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According  to  its  Charter,  the  mission  of  AGARD  is  to  bring  together  the  leading  personalities  of  the  NATO  nations  in  the  fields 
of  science  and  technology  relating  to  aerospace  for  the  following  purposes: 

—  Recommending  effective  ways  for  the  member  nations  to  use  their  research  and  development  capabilities  for  the 
common  benefit  of  the  NATO  community; 

—  Providing  scientific  and  technical  advice  and  assistance  to  the  Military  Committee  in  the  field  of  aerospace  research  and 
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Theme 


The  combination  of  increasing  military  system  and  task  complexity,  in  the  face  of  inherent  human  limitations  has  set  the  stage 
for  development  of  innovative  system  integration  approaches  involving  the  use  of  knowledge  based  technology.  The  field  of 
Artificial  Intelligence  (AI)  is  becoming  solidified  as  a  science  and  the  technological  aspects  are  developing  rapidly.  The 
implications  for  guidance  and  control  are  enormous.  Practical  applications  of  AI  are  critically  dependent  on  advanced 
architectures,  computer  processing  techniques  and  integration  concepts.  Recent  advances  in  digital  computation  techniques 
including  data  base  management,  represent  the  core  enabling  technology  necessary  for  the  development  of  highly  innovative 
design  concepts,  which  will  ultimately  lead  to  major  new  military  capabilities.  Efficient  tactical  information  management  and 
effective  pilot  interaction  are  essential.  Pilot  decision  aiding,  combat  automation,  sensor  fusion  and  on-board  tactical  battle 
management  concepts  offer  the  opportunity  for  substantial  mission  effectiveness  improvements.  Although  real-time  tactical 
military  applications  are  relatively  few  in  number,  several  exploratory  and  advanced  development  efforts  are  underway. 
Practical  military  applications  of  AI  technology  are  of  primary  interest.  Projected  military  capability  enhancements  along  with 
AI  limitations  were  considered.  Operational  implications,  and  critical  design  trade-offs  were  also  emphasized.  This  symposium 
provided  a  timely  forum  for  assessing  the  overall  state-of-the-art  of  AI  applications  in  the  guidance  and  control  area. 

A  round  table  discussion  to  identify  application  issues  and  opportunities  was  held. 
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La  complcxitc  croissantc  des  taches  militaires  et  des  systemes  d'armes  face  aux  hmites  inherentes  aux  capacites  humaines,  a 
prepare  le  terrain  pour  le  developpcment  d'approches  novatrices  recourant  aux  techniques  des  systemes  a  base  de 
connaissances.  Le  domaine  de  I'intclligencc  artificielle  s’affirme  en  tant  que  science  dont  les  elements  technologiques  evoluent 
tres  rapidement.  Les  consequences  pour  le  guidage  et  le  pilotage  sont  considerables.  Les  applications  pratiques  de  I'mtelligence 
artificielle  sont  directement  tributaires  des  ameliorauons  des  architectures,  des  techniques  du  traitement  des  donnees  par 
ordinateur  et  des  concepts  d’integration.  Les  progres  recents  dans  le  domaine  des  techniques  de  calcul  numeriques,  y  comprts  la 
gestion  de  bases  de  donnees,  representent  le  noyau  technobgique  indispensable  au  developperr.ent  de  concepts  hautement 
novatcurs,  qui  deboucheront,  a  terme,  sur  de  nouvclles  et  importantes  applications  militaires.  La  gestion  efficacc  des 
informations  tactiques  et  la  bonne  interaction  piiote-systemc  sont  essentielles.  L’aide  a  la  decision  pour  le  pilote, 
I’automatisation  du  combat,  la  fusion  aes  capteurs  et  les  concepts  de  la  gestion  tactique  de  la  bataille  par  des  moyens  embarques 
ouvrent  la  voie  a  une  amelioration  substantielle  de  l’efficaci'e  operationnelle.  Bien  que  les  applications  temps  reel  soient  encore 
relativement  rares,  un  certain  nombre  de  piojcts  de  developpements,  tant  expioratoires  qu’avanccs,  sont  en  cours  a  I’hcure 
actuelle.  Les  applications  militaires  des  technologies  de  I’mtelligence  artificielle  sont  d’un  interet  primordial.  Les  ameliorations 
attenducs  de  l’efficacite  des  moyens  militaires  ont  ete  examinees  conjointement  avec  les  limitations  previsiblcs  de  (’intelligence 
artificielle.  Les  implications  operationnelles  et  les  compromis  critiques  etablis  lors  de  la  conception  des  systemes  ont  fait  l’objet 
d’une  analyse  particuliere.  Ce  symposium  a  ete  ainsi  1’occasion  pour  faire  revaluation  de  l’etat  de  I’art  des  applications  de 
l’intelligencc  artificielle  dans  le  domaine  du  guidage  et  du  pilotage 

Une  table  rondc  s’est  tenue  dans  le  but  de  faire  emerger  les  applications  potentiellcs 
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L'AVENIR  Dl!  CONTROLS  DE  LA  NAVIGATION  AERIENNE  EN  EUROPE 
LE«MUR  DE  LA  CAPACITY* 

Jacques  Vilhers 


PREMIERE  PARTIE  :  LA  CAPACITY  DE  L’ESPACE  ADRIEN  EUROPEEN 

La  premidre  partie  de  cet  article  examine  done  les  caractdristiques  spdcifiques  de  I’espace  aenen  europden  des  points  de  vue 
opdradonnel,  technique  et  social. 

Elle  explique  pourquoi  I'espace  adrien  europden  est  d'ores  et  ddjh  plus  saturd  que  celut  des  Etats-Ums.  Parmi  les  nombreuses 
causes  figure  l'hdtdtogdnditd  du  systdme  europden,  telle  qu’elle  rdsulte  des  conditions  hislorques  de  son  ddveloppement. 

On  montrera  cependant  que  son  unification  ne  pourra  pas  se  faire  rapidement  et  que,  m  une  telle  dvolution,  ni  celle  qui  pourrait 
rdsulter  de  progrds  techniques  «au  fil  de  l’eau»  ne  seront  &  dies  seules  susceptibles  de  permeltre  un  accroissement  de  capacitd  suffisam 
pour  sadsfaire  en  quantitd  et  en  qualitd  une  detnande  de  trafic  adnen  en  expansion  rapide. 

Les  flux  d’informauon  et  les  flux  de  laches  finissent  par  ddpasser  les  potentialitds  de  traitement  par  des  opdrateurs  humains. 

La  deuxiime  partie  proedde  &  une  analyse  approfondie  de  ces  phdnomcnes  de  saturauon. 

L’dtude  montre  que  les  temps  sont  proches  oh  il  seta  possible  et  indispensable  que  les  calculateurs  puissent  assister  les  controleurs 
dans  leurs  ddcisions,  et  plus  particulidrement  dans  celles  qui  concement  leur  stratdgie  d'acnon  en  temps  rdel. 

L'analyse  met  cependant  en  dvidence  que  cette  dvoluuon  se  heurte  it  une  difficult^  conceptuelle  fondamentale  due  h  un  phdnomdne, 
qui  sera  ddnommd  le  «mur  de  la  capacitd»,  et  qui  rdsulte  des  interactions  entre  le  libre  arbitre  du  controleur  et  celui  du  calculateur. 

La  ddmarche  proposde  pour  parvenir  &  franchir  ce  «mur  de  la  capacitd»  fait  appel  ii  la  mise  en  reuvre  de  «programmes  experts*  et 
suggire  un  projet  dont  on  ddcrira  le  principe  et  la  spdcificitd 

Ce  projet  sera  ddnommd  FREGATE  (Fibre  de  Rdgulation  Ergonomique  de  la  Gesuon  des  secteurs  et  d’Amicipation  des  Taches 
Excesses) ;  il  devrait  permettre  de  faire  dvoluer  d’une  mamdre  continue  1c  systdme  actuel  vers  une  automatisanon  croissante  des 
processus  d’assistance  aux  controleurs  Ce  projet  consume  un  fit  directeur  possible  pour  bs  etudes  et  recherchcs  qui  se  ddveloppent  au 
sein  des  Etais  europdens  et  qui  sont  fdddrdes  par  Eurocontrol  et  par  la  CEE. 

Il  serait  impdratif  que  ces  travaux  ddbouchem  en  temps  opportun  sur  la  conception  d’un  systime  cominun  pour  le  ddbul  du  Slide 
prochain.  Les  analyses  et  propositions  du  prdsent  article  constituent  une  contribution  en  ce  sens  et  ddenvent  une  forme  dventuelle  de 
leurabouussement. 

La  tache  sera  -onguc  et  ardue,  mais  les  enjeux  sont  essentiels 
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CAI  ACITE  DE  L’ESPACE,  CAPACITE  DES  AEROPORTS 

Alorsque  le  transport  adnen  europden  entre  dans  la  phase  acavedeia  nuseen  teuvrede  1’Acte  unique,  lesautontds  responsables  et 
les  compagmes  adriennes  manifestent  une  inquidtudc  croissante  concemant  la  saturation  de  certains  aeroports  et  du  controle  de  la 
circulation  adnenne 

Cette  prdoccupation  est  d’autant  plus  justifide  que  I’effort  de  libdralisauon  se  ddveloppe  dans  un  contexte  de  trds  vive  croissance  du 
transport  adrien,  alors  qu'aux  Etats-Unis,  les  premiircs  anndes  de  la  ddrdglemeniauon  se  sont  dcouldes  sur  un  fond  de  rdcession 
dconomique. 

Or,  pour  s'dpanouir  librement  la  concurrence  suppose  des  infrastructures  largement  dimensionndes  -  tel  n’est  ddjh  plus  le  cas  en 
Europe. 

Les  infrastructures  doivent  d’ailleurs  etre  d’autant  plus  facilcment  accessibles  que  le  libdralisme  s’oppose,  par  sa  nature  meme,  h 
1’optimisation  de  1’utihsation  de  leur  capacitd. 

Qui  plus  est,  l’expdrie'  ce  amdncaine  a  montrd  que  la  capture  &  son  profit  des  capacitds  d’mfrasmicture  disponibles  constilue  pour 
chaque  compagnie  adnenne  un  enjeu  strategique  d’aula  it  plus  ddtermmant  que  la  concurrence  est  plus  vive  et  que  cette  capacitd  est  plus 
Umitde. 

Tel  est  bien  le  mdcanisme  qui  a  accdldrd  d’une  mamdre  spectaculaue  la  saturation  des  adroports  pnncipaux  amdncains  Alors  que  le 
nombre  de  mouvements  totaux  (circulation  adnenne  en  route)  n’a  cru  que  de  26%  de  1978  h  1987,  celui  des  grands  adroports  a 
augmentd  de  64%  pendant  cette  meme  pdriode;  de  surcroit,  les  mouvements  sur  les  grands  adroports  ont  eu  tendance  d  se  concentrcr 
sur  des  pdnodes  de  pointe  pour  optimiser  les  possibilitds  de  correspondances  ( 1 ). 

Un  phdnomdne  de  meme  nature  ne  manquera  sans  dome  pas  de  se  produire  en  Europe  oil  plusieurs  poles  adroportuaire?  sons  ddjd 
prochcs  de  la  Saturation. 

Contrairement  aux  Etats-Unis,  il  se  manifeste  cependant  en  Europe  une  certame  presston  antagomste  due  d  la  montde  en  diversitd  et 
en  puissance  des  lignes  secondaires  desservant  directement  les  villes  de  province 

En  revanche,  la  liberalisation  provoquera  une  intensification  d'un  phdnomdne  ddjd  inquidtant  en  Europe  et  beaucoup  plus  sdvdre 
qu’aux  Etats-Unis:  la  saturation  du  controle  de  la  circulation  adrienne  en  route. 
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Dans  les  causes  des  retards,  la  pan  respective  de  la  saturation  du  controle  en  route  et  de  celtc  des  airoports  est  souvent  mal 
appriciie  par  les  utilisateurs,  dans  la  mesure  oil  les  dilais  aux  dicollages  imposes  par  les  centres  de  controle  en  route  provtennent, 
scion  les  cas,  de  leur  propre  saturation  ou  de  ccllc  des  airoports  de  depan  ou  de  destination. 

Oist  en  giniral  l'ensemble  qui  est  agtigi  dans  l’estimation  de  la  penalisation  apportie  par  le  controle. 

II  serait  auss1  fallacieux  d’attnbuer  la  source  des  problimes  europiens  aux  seules  ltmites  de  capaciti  du  systime  airoportuaire  qu’i 
cellcs  du  seul  contrdle  en  route.  Toutes  deux  imposent  des  contraintes  dont  les  causes  imbriquies  soulivent  u.;s  problimes  de  nature 
tris  diffirente  et  appellent  des  remides  spicifiques, 

11  serait  illusotre  de  traiter  certains  problimes  en  sous-estimant  les  autres,  dans  la  mesure  ou  c’est  it  chaque  instant  le  maillon  le  plus 
faible  qui  limite  la  capaciti  totale  du  systime  dans  son  ensemble;  les  phinomines  de  saturation  ponctuelle  se  propagent  pendant  une 
longue  pin  ode  dans  tout  le  systime:  par  exemple,  la  saturation  d'un  airoport  peut  entrainer  la  saturauon  du  controle  en  route  lorsque  la 
situation  se  dibloque  et  inveisement. 

On  rappellera,  pour  mimoire,  que  la  saturation  des  poles  airoportuaires  risulte  de  la  conjonction  de  facteurs  tris  divers:  capaciti 
des  pistes  et  des  espaccs  airiens  associis,  capaciti  des  aires  de  stationnement  et  des  airogares,  liaisons  ville-airoport,  contraintes 
icologiques... 

11s  soulivent  des  problimes  de  nature  locale,  tant  pour  le  diveloppement  des  infrastructures  que  pour  la  fixation  de  leurs  rigles 
d’utilisation  ,  ces  demiires,  et  notamment  l'allocation  de  crineaux  horaires,  risultent  de  marchandages  bilatiraux  sur  un  fond  de 
droits  acquis  historiques. 

Le  p risen!  article  se  propose  de  ne  traiter  que  le  problime  de  la  capaciti  du  systime  de  controle  de  la  circulauon  ainenne  en  route 

Dans  I’itai  actuel  du  systime  europien,  et  devant  1'inquiitude  soulevie  par  les  sombres  perspectives  qui  risulteraient  de  son 
ivolution  au  «fil  de  l'eau»,  un  effort  de  clarification  semble  en  effet  nicessairc 

Dans  une  premiire  partie,  on  itudiera  les  caractinstiques  spicifiques  du  contrdle  de  la  circulation  airierne  en  Europe  et  les 
amiliorarions  susceptibles  de  lui  etre  apporties  en  consiquence.  On  montrera  cependant  que,  mime  si  celles-ci  pouvaient  toutes  etre 
mises  en  oeuvre,  l’augmcntation  de  la  capaciti  qui  en  risulterait  ne  serait  pas  suffisar.le  pour  permettre  de  faire  face  dans  de  bonnes 
condiuons  k  la  croissance  privisible  du  trafic. 

Dans  une  deuxiime  partie,  on  itudiera  la  nature  profonde  des  micamsmes  de  saturation  (le  «mur  de  la  capacitio)  et  on  proposera 
un  projet  original,  dont  on  peut  espirer  qu’.l  sera  de  nature  &  peimettre  que  le  controle  de  la  circulation  ainenne  ne  constitue  pas  dans 
l’avenir  un  goulct  d'itranglemem  au  diveloppement  du  transport  aenen  en  Europe. 


LA  SATURATION  DU  CONTR6LE  EN  ROUTE  EUROPEEN 

L'ensemble  des  pinalisations  apporties  par  1’insuffisance  de  capaciti  du  contrdle  en  route  et  des  infrastructures  airoportuaires  se 
chiffre  a  plusicurs  milliards  d  icus  par  an  La  fluiditi  du  contrdle  en  route  constituu  un  problimc  majeur  runnel  il  convient  de 
s'attaquer  tisolumcm. 

It  serait  en  effet  illusoire  d’accroitrc  la  capaciti  des  infrastructures  airoportuaires,  si  le  contrdle  de  la  circulation  ainenne  devait  en 
limiter  les  potenualitis  d'utilisation. 

II  est  done  essenuc!  de  recenser  et  d’analyser  tous  les  facteurs  qui  affectent  la  capaciti  du  condole  en  route  europien. 

On  exarmnera  ainsi  successivemcnt : 

-  I'espace  ainen  europien, 

-  I’espace  social  europien, 

•  I’espace  opirationntl  et  technique  europien 


L’ESPACE  A&RIEN  EUROPEEN 


Si  on  compare  sans  nuances  I'espace  ainen  europien  k  I’espace  ainen  amincain,  on  ne  peut  que  s’itonner  du  fait  que  I’espace 
europien  soil  sujet  &  des  saturauons  alors  que  k  trafic  imeneur  amincain  est  beaucoup  plus  intense  que  celui  de  I’Europe. 

Mais  une  telle  comparaison  grossitre  n’est  guire  pertinente. 

La  capac.ti  d’un  systime  de  contrdle  ne  se  mesure  pas  k  1’aune  du  trafic  moyen  ct  de  I’espace  total,  mats  k  celle  du  trafic  de  pomte 
dans  les  er.paces  dtspombles  pour  l’icouier,  li  et  quand  il  se  prisente 

La  repartition  des  axes  i  grand  trafic,  de  meme  one  la  distnbution  horaire  du  trafic  obiisscnt  de  part  ct  d’autre  k  des  lots 
prolondimei.i  duff  rentes :  1  Europe  doit  faire  face  k  des  iwintes  de  trafic  k  certains  moments  et  sur  certains  axes  qui  figurant  parmi  Its 


Le  trafic  europien  se  concentre  essentiellement  pendant  la  joumie :  le  trafic  de  nuit  est  quasiment  inexistant  en  raison  des  couvre- 
feux  impo'is  Sur  tes  airoports  et  de  I’absence  de  trafic  ntineur  long-coumer  comparable  au  «coast  to  coast»  survolant  le  temtoire 
amincairt  De  meme,  les  variations  horaires  et  saisomiiircs  sont  beaucoup  plus  marquees  en  Europe,  notamment  en  raison  de  1’attrait 
des  Europiens  pour  les  voyages  de  vacances. 


De  surcroi',  I’espace  dispomble  pour  achemmer  les  trafics  les  plus  denses  y  est  considirablcment  Iimiti  par  les  grandes 
cgglomiraticns  urbaines  proches  les  unes  des  autres,  qui  ne  peuven.  pas  etre  survolies,  et  par  I’espace  riservi  aux  zones  de  controle 
terminal  cc  icnrs  airoports. 
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Enfin,  il  existe  peu  d’espaces  ii  I’dcart  des  grands  trafics  pour  permettre  !c  ddploiement  des  bases  adriennes  miliiaires  el 
.'emrainement  systdmatique  des  dquipages. 

La  cohabiiation  der  o.  ix  missions  civile  el  militaire  dans  un  meme  espace  limitd  ne  peut  manquer  de  faire  peser  de  iourdes 
contrainies  sur  l'une  comme  sar  1’autre.  La  pdnalisaiion  du  trafic  civil  qui  en  rdsulte  fan  I’objet  de  suffisamment  d'analyses  publiques 
pour  qu'il  ne  soil  pas  ndcessauv  de  s'y  attardcr,  en  revanche,  celle  que  subit  le  systime  militaire  (en  limitations  opdrationnelles  et  en 
cout)  est  moins  connue;  l’accroissement  rapide  du  trafic  tend  &  rendre  les  grands  axes  de  trafic  civil  de  plus  en  plus  difficiles  k  traverser 
par  les  vols  militaires;  ces  demiers  doivent  d’ores  et  ddjit  subir  en  moycnnc  2,2  manoeuvres  de  contoumement  d’avions  civils  par 
hcure  de  vol  controld. 


L’ESPACE  SOCIAL  El'ROPltEN 

Les  consdquences  brutaies  sur  I’dcoulemem  du  trafic  des  mouvements  sociaux  qui  se  produtsent  k  rdpddtion,  ici  ou  lit,  constituent  & 
I'dvidence  la  cause  la  plus  frdquente  et  la  plus  sdvhre  de  limitation  de  la  capacitd  du  systime  europden. 

Dans  I'analy  e  statistique  des  retards  et  des  contrainies  que  ce  systime  occasionne  effectivement  aux  compagmcs  adriennes,  il 
faudrait  done  faire  clairement  la  past  de  ce  qui  revient  &  la  limite  de  sa  capacitd  effective,  et  de  ce  qui  doit  etre  attnbud  aux  consdquences 
de  tensions  sociales  enddmiques. 

C’est  d'ailleurs  prdcisdment  dans  les  pdnodes  de  plus  forte  pression  de  trafic  que  se  mamfestent  les  mouvements  sociaux  :  e'est 
pendant  ces  pdnodes  que  les  contrainies  qui  pdsent  sur  les  personnels  sont  les  plus  vives  (et  done  que  leur  insatisfacuon  est  la  plus 
grande)  et  que  I’impact  sur  le  trafic  d'un  mouvement  de  mauvaise  humeur  ou  de  grhve  est  le  plus  marquant. 

Les  conflits  sociaux  ponem  non  seulement  sur  les  salaires,  mais  aussi  sur  les  conditions  de  travail  qui  s'aggravent  dds  que  le 
systime  appnoche  de  sa  limite  de  capacitd,  notamment  si  cette  limite  rdsulte  de  I'insuffisance  des  effectifs  mis  en  place  pendant  les 
heures  de  pomte. 

Le  problime  est  umvetsel,  mais  il  est  rendu  d'autant  plus  redoutable  en  Europe  qu’un  mouvement  de  grdve  (ou  de  rdduction 
intentionnelle  de  capacitd  dite  «grhve  du  zile»)  dans  un  des  pays,  entraine  des  perturbations  dans  l’ensemble  du  systime  En  revanche, 
une  partie  du  trafic  peut  etre  acheminde  en  contoumant  les  zones  stdnlisdes  par  une  grive. 

Ces  mouvements  s'dtalent  souvent  sur  de  tris  longues  pdnodes  au  cours  desquelles  1’abscncc  de  bonne  volontd  des  controlcurs 
entraine  une  rdduction  de  la  capacitd  effective  du  systime,  sans  publicttd  de  ces  componements  comme  dans  le  cas  d’une  grive 
tranche. 

C’est  ainsi,  par  exemple,  qu’i  la  suite  du  protocole  d'accord  signd  le  4  octobre  1988  cn  France  avec  les  personnels,  la  capacitd  du 
systime  franjais  est  rapidement  revenue  it  un  niveau  tel  qu’elle  n'occasionne  plus  pour  1’instant  de  contrainies  Iourdes  sur  le  trafic. 

Certains  espircnt  que  ces  prablimes  sociaux  pourraient  trouver  leur  soluuon  dans  le  cadre  d’une  fonction  publique  curopdenne  ou 
dans  celui  de  statuts  des  personnels  ddgagds  des  contrainies  mhdrentes  aux  fonctions  publiques  nauonales  ou  europdennes.  II  s'agit  de 
problimes  de  nature  parnculiirement  ddlicate :  le  rythme  devolution  ne  saurait  ddpasser  celui  des  menialitds. 

L'exemple  amdncain  montre  d’ailleurs  que  la  gesuon  d'un  grand  systime  unique  n’est  pas  exempte  de  graves  difficultds  sociales  d 
a  appeld  des  mesures  radicales  qui  ne  sauraient  etre  donndes  comme  exemple  altractif  aux  personnels  europdens. 


L’ESPACE  OPERATIONNEL  ET  TECHNIQUE  EUROP&EN 

Il  est  clatr  que,  malgrd  les  efforts  d’Eurocontrol,  le  systime  europden  reste  caractdnsd  par  un  manque  dvident  d’homogdndttd 
d’exploitauon,  de  planificauon  et  de  conception.  Faut-il  s’en  dtonner  ? 

Cette  hdtdrogdnditd  a  des  racines  profondes  qui  tiennent  notamment  k  un  ensemble  de  causes  historiques.  Chaque  Etat  a  dtd  amend  i 
ddvelopper  son  propre  systime  en  fonction  de  conditions  locales  spdcifiques :  nature  et  intensitd  du  trafic,  organisation  des  services 
publics  nationaux  (civds  et  militaires),  structure  et  culture  sociales,  systimes  de  formation  et  de  sdlection,  mdthode  et  style  d’approche 
des  problimes  opdrationncls  et  techmques. 

De  surcroit,  la  cohabitauon  des  circulauons  adnennes  civile  et  militaire  dans  de  mimes  espaces  implique  des  coordinations  intimes 
entre  les  dtats-majors  et  les  services  el  aggravent  encore  le  caractire  nauonal  de  chacune  des  composantes  civiles  du  systime  europden. 

L’exploitation 

Ces  dispantds  concement  d’abord  I’exploitation  opdrationnelle  elle-meme.  II  existe  dans  ce  domaine,  plus  que  dans  aucun  autre, 
une  oculture  professionnelle*  affumde,  dont  les  paniculansmes  affcctent  profonddment  les  mdthodes  et  les  modalitds  d’exdcubon  du 
controle. 

Lc  controle  de  la  circulation  adnenne  est  certes  une  technique,  mais  e'est  surtout  un  art  pragmatique  dont  la  pratique  exige 
(’acquisition  d'un  «savoir  faire»  et  le  recours  i  tout  un  ensemble  de  rdfiexes,  voire  de  «recettes»  locales;  il  ne  peut  pas  etre  considdrd 
comme  ddcoulam  de  la  simple  mise  en  ceuvre  d’une  thdorie  gdndrate.  Les  hommes,  en  foncuon  de  leur  niveau  de  recrutement  et  de  leur 
mode  de  formation,  acquiirent  des  habitudes  de  travail  dans  lesquelles  les  conditions  locales  jouent  un  role  ddterminant. 

Rappelons  notamment  qu’un  appremissage  et  un  emrainement  «sur  le  tas»  d’environ  trois  k  quatre  ans  sont  ndcessaires  pour 
I’accession  k  la  quahficabon  maximale  de  controle  dans  un  centre  k  grand  trafic. 

pans  de  tellers  conditions,  on  comprendra  que  le  systime  soit  soumis  &  une  tris  grande  inertie  devolution  :  tout  ebangement 
significatif  dans  les  habitudes  de  travail  des  controlcurs  implique  une  modification  des  rdfiexes  psychosensonels  qu’ils  ont  acquis  par 
une  longue  pratique.  Or,  c’est  I’existence  de  ces  rdfiexes  stirs  et  rapides  qui  constitue  la  source  mime  de  la  capacitd  de  pointe  du 
systime. 
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C'est  ainsi  d’ailleurs  que,  lore  d’une  mutauon  d'un  centre  i  un  autre,  meme  au  sein  d'un  meme  systime  national  (voire  d’un  sous- 
ensemble  de  secteurs  it  un  autre  au  sein  d'un  mime  centre),  les  controleurs  doivent  subir  un  nouvel  entrainement  de  longue  durie. 

La  spicificiti  de  ce  miner  est  une  consequence  de  la  nature  heuristique  des  processus  de  contrdle;  il  en  risulte  d'ailleurs  que 
l'automatisadon  mime  partielle  d'un  tel  systime  ne  pourra  pas  progres^cr  notablement  par  la  seule  mise  en  oeuvre  de  processus  logico- 
mathimadques,  et  done  que  le  recours  it  «l'intelligence  artificielle*  -  c’est-it-dire  la  reproduedon  par  des  calculateurs  de  l'cxpemse  et  de 
1‘acquis  des  contrdleurs  ■  consdnie  la  seule  voie  prometteuse  pour  t’avcntr  (2). 

Les  hommes  et  teurs  oudls  de  travail  ne  sont  done  pas  dtrcctement  interchangeables,  dans  le  court  temne,  d’un  pays  it  un  autre,  ni 
mime  d’un  centre  a  l’autrr,  voire  d’un  secteur  i  l’autre, 

Les  consiquences  pour  l’usagrr  des  dispamis  d'exploitauon  se  concritisent  notamment  par  le  fait  que  le  passage  d’une  frondirc 
par  un  avion  est  souvent  moms  bicn  optimise  que  le  simple  passage  d’un  secteur  &  un  autre  au  sein  d'un  systime  nauonal 
(coordinations  plus  lourdes,  espace  moms  bien  organisi).  Dans  cede  mesure,  la  mulnplication  des  exploitations  rationales  juxtaposies 
nuit  effeedvement  it  la  fluidity  du  trafic. 

II  serait  cependant  fallacicux  de  conclure  de  ce  seul  fait  que  le  controle  de  la  circulation  airienne  en  Europe  est  pinalisi  par 
l'existence  d’un  trop  grand  nombre  de  centres  de  controle. 

On  serait  plutdt  tenti  de  penser  que  l’orgamsation  des  mouvements  d’avtons  ripond  plus  it  des  impiratifs  de  lutte  concurrenttelle 
qu’i  une  utilisation  optimale  des  ressourees  rares  constuuies  par  les  capacitis  airoportuaires  et  les  espaces  airiens,  et  qu’en  ce  sens,  il 
existe  peut-ctre  trap  de  compagmes  airiennes  en  Europe! 

Le  choix  du  nombre  optimal  de  centres  de  contrdle  consume  un  problime  d’une  autre  nature;  il  doit  procider  d’une  analyse  portant 
sur  un  grand  nombre  de  paramitres  techniques,  iconomiques  et  sociaux.  C'est  ainsi,  par  exemple,  que  les  autoritis  franpaises  ont  iti 
amenies  it  ripartir  progresstvement  le  controle  en  route  entre  trois,  puis  cinq  centres  distincts,  ce  qui  n’a  pas  nui  pour  autant  it 
I’efftcaciti  de  l'ensemble  du  systime. 

A  I’heure  de  la  tili-informauque  et  du  Transpac,  l'infoimation  circule  aussi  facilement  entre  des  centres  giographtquement  siparis 
qu’au  sein  d'un  mime  centre. 

Toutes  les  positions  de  controle,  oil  qu’elles  soient  localisies,  sont  d’ailleurs  alimenties  en  information  de  base  (plans  de  vol, 
radar,  radiotiliphonie  air-sol,  tiliphone)  par  voie  de  liaisons  tiliphoniques:  il  y  a  maintenant  bien  longtemps  que  les  controleurs  ne 
commumquent  plus  directement  entre  eux,  que  dans  des  circonstances  rclanvement  exccpuonnelles,  meme  au  sein  d’un  mime  centre. 

L’efficaciti  globale  du  controle  dicoule  en  revanche  du  nombre  de  secteurs  de  controle  entre  lesquels  est  dicoupi  l’espace. 

Le  nombre  de  secteurs  nicessaires  -  e’est-it-dire  le  nombre  d’unitis  de  controle  autonomes  coordonnies  deux  a  deux  -  ne  risulte 
que  du  nombre  maximal  d’avions  qu’un  controleur  et  ses  assistants  peuvent  prendre  en  charge  simultaniment  dans  un  environnement 
donni. 


Or,  toutes  choses  igales  par  ailleurs,  ce  nombre  maximal  -  de  l’ordre  de  10  it  IS  avions  scion  la  nature  du  secteur  -  est  le  mime 
dans  tous  les  pays ;  il  est  en  particulicr  le  mime  en  Europe  et  aux  Etats-Ums. 

En  revanche,  le  nombre  moyen  d’avions  qu’un  controleur  achemine  effeedvement  au  cours  d’une  joumic  ou  d'une  annie  peut  etre 
tris  diffirent  d’un  lieu  it  un  autre,  car  il  est  dtrcctement  function  de  la  ripartition  du  irafic  dans  le  temps.  Avcc  ce  seul  entire,  le 
systime  amincain  peut  sembler  beaucoup  plus  «productif»  que  le  systime  europien,  car  le  trafic  y  est  soutenu  d’une  maniire  plus 
continue.  Mais  la  productiviti  mesurie  par  ce  entire  dipend  aussi  beaucoup  des  conditions  de  travail  (nombre  d’heures  de  controle 
effecuf  par  semaine,  organisation  du  travail,  flexibility  des  horaires,  congis  octroyis  pendant  les  jours  de  pointe,  etc.). 

Surce  dernier  point,  le  systime  amincain  est  sans  aucun  doute  plus  «productif»  que  cclui  de  I’Europe  en  giniral  on  notera  par 
exemple  qu  en  Europe,  une  pardc  de  1  cffccof  est  autonsie  it  prendre  ses  vacances  pendant  les  jours  de  pointe,  alors  que,  au  contraire, 
les  centres  amiricains  restent  dotis  de  l’effcctif  total  qui  est,  de  surcroit,  inviti  it  effcctucr  des  heures  supplimentaires  pendant  ces 
pinodes  les  plus  chargies. 

11  est  oonc  exact  que,  en  moyenne,  un  contrdleur  europicn  achemine  annuellement  moms  de  trafic  qu'un  controleur  amincain  roais 
le  nombre  de  centres  de  controle  n’y  est  sans  doute  pas  pour  grand  chose. 

Il  est  exact  aussi  que  la  «productiviti»  des  controleurs  amincams  (nombre  d’avions  traitis  par  an)  a  cru  d’une  maniire  rapide  aux 
Etats-Unis  (elle  a  doubli  en  10  ans). 


En  Europe  cette  *productiv'.te>*  restc  relativement  constantc  en  moyenne ;  on  notera  cependant  qu’elle  a  cru  brusquement  it  la  suite 
de  la  repnse  btutale  du  trafic  qui  a  surpns  bien  des  plamficateurs.  11  en  a  d’ailleurs  risulti,  par  le  processus  diem  ci-dessus,  a  la  fois 
des  contraintes  accrues  sur  les  personnels  et  des  mouvements  sociaux  qui  ont  amplifii  le  phinomine  de  saturauon. 

En  ce  qui  mceme  les  coflts  salariaux,  les  conditions  sont  tris  diffirenciies  au  sein  de  l’Europe,  aussi  bien  pour  les  mveaux  de 
salaire  (et  des  charges  sociales  et  fiscales  assocties)  que  pour  les  conditions  de  travaii  (nombre  d’heures  effecuves  de  controle  annuel, 
notamment  pendant  les  piriodes  de  pointe  de  trafic,  ficxibiliti  des  horaires. . .). 


La  reduction  de  ces  dispantis  ne  manquera  pas  de  soulever  de  graves  problimes  et  exigera  beaucoup  de  temps  et  de  sages-se,  il 
n’est  d'ailleurs  pas  certain  que  le  cout  total  du  controle  y  trouvera  son  compte. 

L’exemple  amincain  montre  aussi  qu’un  grand  systime  de  contrdle  umfii  est  tris  vulnirable  et  n’est  pas  it  i’abn  de  graves 
problimes  sociaux. 


En  revanche,  l’efficaciti  d’ensemble  de  1’exploitation  europienne  pounait  facilement  progresser  grace  a  l’accroissement  des 
contacts  sous  toutes  leurs  formes  entre  les  systimes  nationaux  :  formation  de  base  commune,  ichange  de  controleurs  et  de  cadres, 
approfondissemem  en  commun  de  tous  les  problimes  d’exploitauon  (organisation  de  1'espace,  methodes  de  controle,  outils  de  travail 
du  controleur,  coordinations  entre  centres. ..). 

La  situation  ivolue  d'ailleurs  favorablement  en  ce  sens.  Elle  pounait  etre  largement  amiliorie  s’d  itait  fait  appel  systimatiquement 
a  des  micamsmes  d’arbitrage  pour  trancher  entre  Etats  ou  centres  voisins  en  cas  de  conflits  opirationnels. 
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11  serait  important  que  soil  dlabord  au  prdalable,  en  cooperation  avec  les  compagnies  adriennes,  un  «Livre  blanc»  recensant  tous  les 
probidmes  d’exploitation  qut  rdsultcnt  spdcifiqucment  de  la  disparity  des  modes  d'orgamsation  de  l’espace  et  d’exploitation  des 
diffdrents  systdmes  de  controle  nattonaux. 

11  est  enfin  un  domaine  oil  l'acuon  commune  s’est  ddjd  imposde  d  l’dvidence,  4  savotr  le  controle  des  flux  de  trafic  pour  rdguler  la 
demande  en  fonction  de  I’offre  de  controle  disponible  et  dvi.er  ainsi  de  satuner  les  centres  de  controle  au  detriment  de  la  sdcuntd  et  de  la 
fluiditd  du  trafic. 

Contrairement  1  celles  du  contrfile  de  la  circulation  adnenne  proprement  dite,  les  relations  de  coordination  de  «flow  control*  ne 
sont  pas  uniquement  de  nature  bdaterale  de  secteur  d  secteur  mais  tmpbquent  la  prise  en  compte  simultanec  de  1’ensemble  du  systdme 
(en  route  et  aeroportuaire)  et  sur  un  trds  grand  volume  d’espace. 

On  ne  peut  done  que  se  fdliciter  du  fait  que  les  meii'es  soient  ddsormais  pnses  pour  mettre  en  place  en  Europe  un  dispositif  de 
•flow  control*  unifid.  Sa  mise  en  ceuvre  souliver?  peut  etre  des  difficulty  moindres  (inais  la  preuve  n’en  est  pas  encore  appor.de)  que 
celles  qui  affectent  le  contrdle  proprement  dit,  notamment  parce  que  cette  pratique  n'a  pas  demdre  elle  une  longue  histoire  diffdrencide 
au  sein  de  chaque  Etat  europden. 

la  part  des  processus  logico-mathdmatiques  moddlisables  est  sans  doute  plus  grande  que  celle  du  contrdle  lui-meme,  ll  ne  faudrait 
cependant  pas  sous-estimer  pour  autant  l’importance  du  «savoir  faire*  spdcifique  qu’ont  adveloppd  par  la  pratique  les  agents  chargds 
de  ce  travail  depuis  plusieurs  anndes. 

On  verra  par  ailleurs  dans  la  suite  du  pidsent  aiucle  que  la  rdgulation  efficace  des  flux  peut  non  seulement  rdduire  les  contraintes 
globales  des  compagnies  adnennes,  d  capacitd  donnde  du  systdme  de  contrdle,  mais  participe  d’une  manidre  non  ndgligeable  d 
l'augmentation  de  la  capacitd  de  pointe  effective. 

La  planification 

C'est  I’OACI  qui  dlabore  les  normes  et  pratiques  rccommanddes  en  matiere  de  navigation  adnenne,  tandis  que  son  groupe 
spdcialisd  (le  GEPNA)  dnonce  les  besoms  opdrationnels  spdcifiques  de  l’Europe  et  cooidonne  la  planification  de  ses  infrastructures  et 
de  ses  moyens  de  contrdle. 

De  son  cotd,  Eurocortro!  contnbue  &  I'harmomsation  de  1'cr .  amble  du  dispositif  europden  el  de  son  ddveloppement  progressif,  et 
assure  la  facturation  commune  des  rcdevances  propres  d  chaque  systdme  national. 

Les  compagnies  adriennes  participent  aux  discussions  au  sein  de  ces  entites. 

Mais  la  planification  effective,  e'est-d-dire  le  financement,  le  choix  et  I’installauon  des  dquipements  ainsi  que  la  mise  en  place  des 
personnels  itstent  sous  l’autoritd  des  instances  nationales. 

Dans  ces  conditions,  on  ne  saurait  escompter  que  la  planification  et  1’adaplation  progressive  et  haimomeuse  du  systdme  aux 
besoms  prdsentent  une  homogdnditd,  une  cohdrence  et  une  efficacitd  aussi  grandes  que  si  dies  relevaient  d’une  autontd  effective 
unique. 

Le  probldme  n’est  cependant  pas  aussi  simple  qu’il  pourrait  le  paraitre  a  pnon. 

Le  pluralisme  des  modalitds  nationales  d’exploitation  opdrationnelle  qui,  on  l'a  vu,  n’est  pas  aisdment  ni  rapidement  rdductible,  ne 
permet  gudre  une  planification  et  une  mise  en  oeuvre  ngoureusement  centraJisdes. 

Par  ailleurs,  les  systdmes  nationaux  sont  soumis  d  des  proeddures  budgdtaires  propres  d  chaque  pays ,  certains  d’entre  eux  ont 
cependant  pu  alldger  les  contraintes  qui  rdsultent  de  la  ngueur  gdndrale  qui  pdse  sur  les  budgets  des  Elats,  en  mettant  en  place  des 
structures  mieux  approprides  (budget  annexe,  dtablissement  public.  .)  qui  permettent,  de  surcroit,  de  recourir  d  des  emprunts  gagds 
sur  les  redevances  d  venir. 

Les  Etats  rencontrent  enfin  des  difftcultds  pour  1'embauche  suffisame  et  en  temps  opportun  des  personnels  ndcessaircs. 

Ces  contraintes  fmancidrcs  sur  la  planification  sont  d’autant  plus  irritantes  qu’il  ne  s’agit  que  de  probldmes  de  structure 
administrative  et  non  de  financement  proprement  dit,  puisque  toutes  les  ddpenses  affdrentes  au  controle  de  la  circulation  adnenne  sont, 
in  fine,  couvertes  intdgralement  par  des  redevances  approprides. 

Certaines  insuffisances  et  lentcurs  d’investissement  qui  ddcoulent  de  ces  contraintes  artificielles,  provoquent  des  points  noirs  dont 
l’cffet  se  rdpercute  sur  la  capacitd  de  l’ensemble  du  systdme. 

Dans  ces  conditions,  il  apparaitrait  souhaitable  que  les  maillons  les  plus  faibles  relevant  de  cette  cause  spdcifique  fassent  aussi 
l’objet  d’un  recensement  systdmatique. 

11  serait  ainsi  judicieux  que  soil  constitud  un  fonds  europden  pour  procurer  les  financements  compldmentaires  aux  Etats  qui  ne 
s'estimeraient  pas  en  mesure  de  proedder  rapidement  aux  amdliorations  indispensables ;  un  tel  fonds  pourrait  dtre  alimentd  par  des 
redevances  communautaires  spdcifiques  4  la  charge  des  compagnies  adriennes,  comme  le  sont  les  redevances  nationales. 

Par  ailleurs,  le  coOt  de  I’ensemble  du  systdme,  de  meme  que  la  rapiditd  de  mise  en  place  de  cenams  matdriels,  pourraient 
avantageusement  bdndficier,  dans  certains  cas,  de  recherche/ddveloppement  coordonnds  et  d'achats  groupds.  Une  telle  politique  aurait 
pour  avantage  compldmentaire  d’inciter  d  la  formation  de  consortiums  europdens  susceptibles  d’affermir  la  position  ddj d  enviable  de 
cenains  industriels  europdens  dans  ce  domaine  paruculier ;  elle  ne  pourrait  cependant  porter  que  sur  les  matdriels  normalisds 
(notamment  les  radars  et  leurs  systdmes  de  traitement  de  1’infotmation  directement  associds,  les  moyens  de  navigation  et  de 
communication,  et  certaines  visualisations),  d  I'exclusion  des  dldments  trds  lids  aux  spdcificitds  de  1’exploitaaon  opdrationnelle  uc 
chaque  pays. 

Le  cas  des  calculateurs  et  de  leurs  logiciels  mdrite  un  examen  paruculier  en  raison  de  la  trds  grande  disparitd  des  systdmes 
nationaux  actuels.  11  serait  souhaitable  qu’d  1’occasion  de  chaque  renouvellement  de  matdriels,  des  directives  europdennes  puissent 
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onemcr  le  choix  dcs  calcuiateurs  ou  pour  le  moins  de  li  rrs  systdmes  d’exploilalion  et  dcs  langages  dvolues  utilises,  de  fapon  a 
prdparer  l’avenir  et  permettre  notamment  l'dchange  ultdrieur  de  parties  croissantes  de  logiciels 

Certains  logiciels,  notamment  de  traitement  de  l’tnformation  Radar  ou  de  plans  de  vol  mdrrteraient  ausst  d'etre  normalises  &  cette 
occasion.  II  faut  cependant  se  garder  de  I’espoir  utopique  de  pouvotr  rempiacer,  dans  un  avenir  proche,  I'essenttel  des  logiciels 
propres  aux  systdmes  nationaux  par  un  logiciel  commun,  tant  sont  indissocrables  les  spdcifici’ds  opdrationnelles  de  ces  systdmes  et  les 
programmes  des  logiciels;  de  surcroit,  les  logiciels  en  service  ont  dtd  profonddment  influrncds  non  scul'-ment  par  les  diffdrences  de 
spdcifications  opdrationnelles,  mais  aussi  par  les  processus  diffdrents  qui  ont  prdsidd  it  leur  dlaboration. 

L’objectif  de  pouvoir  procdder  It  l’unification  des  calcuiateurs  et  des  logiciels  du  systdme  europden  ne  peut  done  qu'etre  reportd  & 
un  avenir  lointain ;  il  implique  une  umficanon  prdalable  de  la  conception  meme  du  systdme  futur. 

La  conception  du  systEme 

Ce  probldme  recile  d  son  tourde  trds  grandes  difficultds. 

U  serait  en  effet  impensable  qu’un  systdme  de  controle,  et  notamment  la  ddfinition  des  outils  de  travail  et  des  processus  de  controle, 
puissent  dtre  conijus  sans  la  participation  active  des  contrdleurs  eux-mdmes  et  sans  la  pnse  en  compte  de  leurs  rdflexes  acquis ;  or,  on 
a  montrd  que  ces  demiers  manquent  manifestement  d'unifonmttd. 

U  n'en  rdsulte  pas  pour  autant  que  le  systdme  europden  sou  condamnd  d  rester  figd  pour  toujours  dans  des  particularismes  et  dans 
un  conservatisme  ddnuds  dc  justification.  Mats  il  cn  rdsulte  sans  aucun  doute  que  les  dvolutions  souhaitables  et  notamment 
l’unification  du  systdme,  ne  pourront  qu’etre  progressives:  il  faut  rejeter  comme  lrrdaliste  l’espoir  de  pouvoir  rempiacer  le  systdme 
diversifid  actuel  par  un  nouveau  systdme  con?u  a  priori  dans  le  silence  des  cabinets  et  des  bureaux  d’experts  ou  rdsultant  des 
compromis  «camdliens»  (3)  de  commissions  et  de  comitds  pan-europdens  et  a  fortiori  de  solutions  venues  d'atllcvrs. 

Voild  sans  doute  bien  un  domaine  ou  la  recherche  du  «plus  petit  commun  mulliple»  des  systdmes  existants  pourrait  bien  se  rdvdler 
comme  le  plus  grand  diviseur  des  dnergies  et  de  l’efficacitd. 

De  grands  espoirs  peuvent  en  revanche  etre  placds  dans  une  future  et  lointaine  gdndration  de  systdmes  de  conception  nouvelle  et 
commune,  dans  lequel  les  automatismes  commenceratent  d  intervenir  directement  et  significativement  dans  les  processus  de  controle 
proprement  dit. 

On  rappellera  que  dans  les  systdmes  en  service  actuellement,  ou  en  cours  de  modernisation,  le  traitement  automauque  ne  porte  que 
sur  la  transmission,  la  corrdlation,  la  mise  en  forme  et  la  visualisation  opnmales  des  informations  pertinentes  d  chaque  instant  pour 
I'exercice  du  controle,  d  l'exclusion  de  toute  participation  d  la  pnse  de  ddcision,  e’est-d-dire  au  controle  proprement  du. 

C'est  done  cette  phase  lointaine  qu’il  faut  dds  maintcnant  prdparer  en  commun.  II  en  est  encore  temps,  car  aucune  rdaltsation 
pratique  en  Europe,  ou  ailleurs,  n’a  encore  permis  dc  ddmontrer  sa  faisabtlrtd,  m  meme  sa  contribution  certaine  d  l’augmentation  de  la 
capacitd  effective  de  l'espace  adnen. 

La  conception  en  commun  du  systdme  hybnde  controlcur/calculateur  pour  le  ddbut  du  sidcle  prochain  da  rail  done  consmuer  le 
grand  projet  mobilisateur  de  1’imagination,  de  la  crdativitd  et  dc  l’dnergie  de  1’Europe  du  contrdle  de  la  circulanon  adrtenne. 

Fort  heureusement,  il  existc  d’ores  et  ddjd  en  Europe  une  importante  activitd  dc  recherche  el  d’expdnmentation,  coordonnde  et 
fdddrdc  dans  le  cadre  du  projet  PHARE  d'Euroconirol  el  de  la  CEE ,  I’Europe  n’a  nen  d  envier  cn  ce  domaine  aux  autres  pays 
ddvcloppds,  meme  d  ceux  qui  ont  le  priviidge  d'exploiter  un  systdme  portant  sur  un  trds  grand  volume  d’cspace  gdrd  par  une  autoritd 
unique. 

Cet  effort  porte  notamment  sur  des  visualisations  nouvelles  plus  «inie)ligentes»  suscepnbles  de  faciliter  le  travail  des  conlroleurs  et 
leur  dialogue  avec  le  calculateur  (cf.  cn  particulier  le  projet  PHIDIAS  du  CENA  en  France). 

Des  moyens  technologiques  nouveaux  som  ddsormais  dispombles  (Radar  mono-impulsions,  liaisons  coddes  air/sol  dues  «Radar 
mode  S»,  calcuiateurs  de  navigation  des  avions  dits  FMS)  A  eux  seuls,  ils  ne  permctlront  pas  l’accroissement  de  la  capacitd  de 
l’espace.  si  on  ne  parvient  pas  d  les  unliser  dans  un  cadre  d’automatisation  du  controle  plus  avanede. 

Four  y  parvemr,  il  est  ndeessaire  de  bien  connaitre  au  prdalable  la  nature  profonde  des  mdcamsmes  de  saturation  de  l’espace. 

Les  nouvelles  phases  d’automatisation  vont  imposer  de  pdndtrer  dans  le  domaine  encore  peu  ddfnchd,  et  d'une  immense 
complexitd,  de  la  coopdration  de  1’homme  et  des  calcuiateurs  dans  des  processus  de  ddcision  en  temps  rdel. 

Il  en  rdsulte  un  probldme  «de  la  poulc  et  de  J’auf»:  nul  ne  se  presse  pour  rendre  obligatoires  ces  nouveaux  moyens  (modes  S, 
FMS)  tant  que  le  systdme  de  contrdle  n’aura  pas  dtd  automatisd  plus  en  profondeur ;  inversement  l'automatisation  de  certains 
processus  de  controle  continuera  &  marquer  le  pas  tant  qu'il  n'aura  pas  dtd  prouvd  que  des  amdhorations  de  capacitd  pourront 
effeenvement  en  rdsulter. 

Or,  chacun  de  ces  processus  demande  cinq  &  dtx  ans.  Pour  sortir  de  ce  cercle  vicieux,  il  devient  impdratif  it  la  fois  de  rendre 
obligatoires  &  court  tetme  dans  les  espaces  congestionnds  les  nouveaux  dquipements  au  sol  et  d  bord  et  de  lancer  d’urgence  un  projet 
d  automadsation  susceptible  d'accroitre  la  capacitd  de  l'espace  avec  les  moyens  actuels  d'abord,  pour  ouvnr  ensuite  la  voie  d  des 
progrds  encore  plus  substantiels  dds  lors  que  ces  dquipements  se  gdndraliseront. 


I  A  CAPACITY  DE  L’ESPACE  A&R1EN  EUROPEEN  :  CONCLUSION 


La  capacitd  du  systdme  de  contrdle  europden  recdle  encore  de  bonnes  potcntialitds  de  progrds  pouvant  rdsulter  des  reductions  de  ses 
disparitds  d’exploitation,  d  planiftcation  et  de  conception. 

A  elles  seules  cependant,  ces  rdserves  et  le  rythme  de  leur  mobilisation  ne  seront  certainement  pas  d  la  hauteur  des  besoins 
nouveaux  de  capacity  qui  r£$ulteront  de  la  croissance  prdvisiblc  du  trafic  europden. 
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Les  limitations  de  capacity  provienncnt  en  effet  dc  ce  que  chaque  avion  et  chaque  patre  d’avions  en  conflit  entre  eux,  ginirent  un 
flux  dc  tfiches  dont  l'enscmblc  ne  doit  &  aucun  moment  dipasser  la  capaciti  de  tratiement  que  les  hommes  peuvent  assumer. 

Parmi  ces  taches,  figurent  non  seuiement  des  actions  dc  nature  intellectuelle  ou  matinclle,  mais  ausst  des  mimorisations  dont 
I’ensemble  ne  doit  jamais  outrepasser  les  aptitudes  du  cerveau  humatn  dans  le  bref  temps  lrnparn  dont  tl  dispose  pour  les  accomplir. 

Au  flux  d’avions  est  ainsi  associd  un  flux  d'information  qui  peut  lui-meme  dipasser  les  limites  des  possibilitds  d’apprdhcnsion,  de 
memorisation  et  de  traitement  du  cerveau  humatn;  en  ce  sens,  il  est  loin  d’etre  dvident  que  I’accroissement  considerable  de  la  richesse 
des  informations  qui  vont  devenir  dtspombles  grace  au  mode  S  soil  en  lui-meme  porteur  de  progris.  Sans  l’assistance  des 
calculatcurs,  ces  informations  compldmentaires  pounraient  au  mieux  etre  mal  utilisdes  et,  au  ptre,  contnbuer  a  saturer  encore  plus  les 
controleurs. 


LE  MGCANISME  DE  LA  SATURATION  DE  L’ESPACE 
L’accumulation  des  taches 

Lorsque  le  controleur  est  soumts  it  une  accumulation  de  taches  qui  approche  de  sa  capaciti  maximale,  et  a  fortiori  s’d  anucipe  que 
le  volume  de  taches  va  augmenter  dans  les  moments  qui  vont  suivre,  il  tend  tout  naturellement  a  accroilre  ses  marges  dc  sdcuntd  et 
done  4  dimtnuer  la  capacitd  effective  de  1’espace  qu’il  controle 

II  cherche  en  effet  4  se  prdmumr  4  tout  pnx  contre  I’dventualitd  d’avoir  4  faire  face  4  des  accumulations  de  taches  qu’il  ne  pourrait 
pas  assumer,  il  cramt  aussi  que,  dans  de  telles  conditions,  ses  performances  de  mdmonsation  soient  amoindrtes  et  que,  en 
consequence,  les  risques  s’accroissent  d’oublier  de  trailer  certames  laches  et  notamment  des  conflits  potentiels  non  encore  rd solus 
entre  panes  d’avions. 

Ce  processus  explique  la  brutalitd  du  phdnomine  de  saturation  dis  lors  que  le  nombre  d’avions  sur  un  secteur  atteint,  ou  nsque 
d’atteindte,  une  Iimite  maximale. 

Les  «filets  de  sauvegarde»  .  le  maillon  manquant 

On  comprend,  dans  ces  conditions,  fintdret  qui  s’attache,  tant  pour  des  raisons  de  sdcuntd  que  de  capacity,  4  la  mise  en  ceuvre  de 
«filets  de  sauvegarde». 

II  existe  actuellement  deux  disposiufs  de  cette  nature  en  amont  et  en  aval  du  controle  propremeni  dit 

En  amont,  on  a  ddjti  dvoqud  ci-dessus  I’impact  positif  sur  la  capacitd,  de  la  rdgulation  des  flux  d’avions  qui  prdmumt  le  controleur 
contre  un  flux  de  trafic  ddparsant  ses  aptitudes  4  l’assumer  en  toute  sdcuntd. 

En  aval,  tl  a  did  mis  en  service  depuis  plusieurs  anndes  un  «filet  de  sauvegarde  Radar»  qui  ne  consume  pas  en  soi  une  aide  au 
controle  (et  qui  ne  doit  pas  etre  utilisd  comme  telle)  mais  qui  ddclenche  une  alarme  ultime  avant  qu’une  collision  due  4  une  ddfaillance 
du  controle  ne  risque  de  sc  produire  effecuvement,  et  avec  un  prdavis juste  ndeessatre  pour  une  intervention  d’urgence. 

Ces  deux  disposiufs  participant  done  directement  4  I’accroissement  de  la  security  et  mdirectement  4  cclui  de  la  capacitd  par  la 
«sdcunsation»  des  contrdleurs. 

Entre  ces  deux  filets  de  sauvegarde  en  amont  et  en  aval,  il  existe  un  maillon  manquant,  il  s’agirait  d’un  «filet  dc  sauvegarde® 
susceptible  de  prdmunir  lc  controleur  contre  I ’accumulation  excessive  4  court  terme  dc  taches  susceptible,  faute  d’action  par  lui-meme 
et  en  temps  utile,  de  le  placer  ultf  neunernent  dans  une  situation  tnconfortable,  voire  dangereusc- 

Mais,  contratrement  aux  deux  «filcts»  amont  et  aval,  un  tel  dispositif  pourrait  non  seuiement  gdndrer  des  alarmes,  mais  participer 
d’une  maniire  directe  au  controle  propremeni  dit  dans  une  de  ses  composantes  importantes:  1'  .donnancement  des  taches. 

Telle  est  l’tdde  directnce  du  projet  dont  les  grandes  lignes  seront  ddentes  dans  la  deuxtime  parue  et  qui  devrait  4  la  fois  constituer 
une  aide  au  controle  et  une  sdcunsauon  compldnientaire  des  controleurs. 

La  generalisation  des  «filets  de  sauvegarde®  sur  tous  les  «filtres»  successes  de  controle  (4)  aurait  ainsi  pour  effet  de  transformer 
l’ensemble  du  processus  de  controle,  actuellement  en  «boucle  ouvene®,  en  un  processus  plus  stable,  plus  efficace  et  plus  stir,  en 
«boucle  fermde®. 

LE  MUR  DE  LA  CAPACITY 

On  devrait  pouvoir  en  escompter  une  augmentauon  effective  de  la  capacitd  de  chaque  secteur  et  done  de  I’espace 

Dans  le  cas  du  controle  tel  qu’il  est  exered  actuellement,  le  controleur  est  amend  4  prendre  des  decisions  «stratdgiques»  qui 
antietpent  un  certain  nombre  de  decisions  qu’il  sera  amend  4  prendre  ultdneurement  en  fonetton  de  Involution  de  la  situation.  Le 
systime  est  stabilise  par  le  controleur  lui-meme:  il  est  le  seu!  maitre  du  jeu. 

lien  serait  de  mime  dans  le  cas  d’un  contrdlc  supposd  etre  entiirement  automauque  etdans  lequel  le  s<  il  «hbre  arbitre®  serailcelui 
du  calculates.  Cependant,  mcme  si  un  tel  projet  dtait  realisable  du  point  de  vue  technique,  opcrationnel,  psychologique  et  social,  il 
n  en  resterait  pas  moins  ndeessatre  d’assurer  une  longue  transition  avee  le  systime  manuel 

Le  problime  du  passage  par  un  systime  hybride  homme/calculateur  est  done  incontoumable. 

Or,  tout  systime  dans  lequel  lc  calculates  sera  amend  a  participer  4  I  ’elaboration  des  decisions  soulive  un  problime  redoutable: 
pour  que  I’assistance  4  la  decision  foumie  par  le  calculates  sou  efficace,  il  faudrait  que  le  calculates  soil  informi  non  seuiement  des 
choix  retenus  par  le  controleur  parmi  les  propositions  qu’il  lui  prisente,  mais  aussi  qu’il  puisse  anticiper  I’ensemble  des  choix  que  le 
controleur  effectuera  ultdrieurement  au  fur  et  4  messe  de  involution  de  la  situation. 
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En  effet,  tout  chotx  nouveau  peut  impliqucr  une  modification  de  la  situation  d'ensemble  et  obliger  ainsi  de  revenir  sur  des  choix 
antdrieurement  effectuds.  Le  systime  a  done  toutes  les  chances  de  «pomper»,  c’est-ii-dire  de  i  e  pas  trouver  sa  stabilitd  en  raison  de  la 
cohabitation  de  deux  libres  arbitres  (5),  meme  si  e’est  l’un  d'eux  (I'homme)  qui  prend  seul  les  Idcisions  finales. 

La  transition  du  systime  actuel  vers  un  systime  oil  I’homme  sera  assistd  dans  ses  ddeisiens  oar  la  machine  se  heurte  done  i  un 
phdnomine,  instable  par  sa  nature  meme,  qu’on  ddnommera  le  «mur  de  la  capacitd*,  par  anal  >gie  aux  changements  brusques  qui  se 
manifestent  lors  du  passage  du  «mur  du  son»,  i  la  difference  pris  cependant  que  le  systime  de  contrOle  devra  demeurer  durablenicnt 
dans  une  zone  de  transition. 

La  faisabilitd  d’un  systime  assurant  cette  transition  d'une  maniire  autostable  et  susceptible  devolution  progressive  vers  une 
automatisation  croissante  n’est  pas  encore  ddmontrde;  ll  n'est  pas  possible  non  plus  it  ce  stade  de  prdvoir  l’ampleur  de  sa  contribuuon  h 
l’accroissement  dc  la  capacitd  effective  des  secteurs  de  controle. 

Le  projet,  dont  on  trouvera  les  grandes  lignes  dans  la  deuxtime  panie  de  cette  contribution,  n'dchappe  pas  &  cette  difficultd 
conceptuelle  et  d  ces  incertitudes ;  il  se  caractdnse  cependant  par  le  fait  qu’il  propose  une  straidgie  et  des  moyens  spdcifiquement 
imagmds  pour  tenter  de  les  sutmonter. 


DEUXI&ME  PART1E :  UN  PROJET  POUR  LE  PASSAGE  DU  MUR  DE  LA  CAPACITY 
LA  SPECIFICITY  DU  PROJET 
LES  BASES  THfiORIQUES 

Les  bases  d’un  tel  projet  sont  constitute  par  un  ensemble  thdorique  dejd  cud  ci-dessus  en  rdfdrcnce 

Les  rdsultats  encourageants  des  prenudres  realisations  effectudes  sur  ces  fondements  d  I’Ecole  Nationale  de  I’Aviation  Civile 
(ENAC)  par  Leroux  et  Genthon  (1986/87)  puis  Leroux  ct  Alliot  (1987/88)  permeitent  ddsormais  d’envisager  des  prolongements 
pratiques  et  de  formuler  les  grandes  lignes  d’un  projet  prdcis. 

LA  SPECIFlClrt  DU  PROJET 

Ce  projet  se  caractdnse  par  1 ’ensemble  des  caractdnsuques  suivantes  :■ 

a)  Le  controlcur  reste  seul  responsable  du  controle 

b)  Le  controlcur  est  seul  maitre  de  toutes  ses  actions,  y  compris  de  route  manipulation  (de  clavier  ou  autre)  qui  ne  saurait  done  d  aucun 
moment  lui  etre  imposde  (done  pas  de  taches  suppldmentaires  induites  par  les  processus  lids  d  I ’automatisation). 

c)  Le  calculates  effectue  toutes  les  opdranons  de  contr6le  en  paralltle  avec  le  contrdleur  mais  inddpendamment  de  celui-ci.  Le 
processus  automatique  n'est  qu’un  processus  latent  en  ce  sens  que,  contrairement  au  controleur,  le  calculates  ne  passe  jamais  lui- 
meme  d  faction,  ni  n’impose  ses  solutions 

d)  Le  processus  automatique  ne  preseme  au  controlcur  le  rdsuitat  de  son  activitd  parallilc  que  : 

-  pos  lui  suggdrer  une  action  spdcifiquc  de  contrflle  qu'il  juge  immddiatement  opponune  ou  urgente. 

■  pour  lui  prdsenter  une  situation  d’ensemble  fondde  sur  des  informations  agrdgdes  dlabordes  ps  le  «processu$  dc  controle 
automatique  paralldle»  et  correspondent,  d  chaque  moment  et  d’une  manidre  aussi  rdaliste  que  possible,  aux  «agrdgats 
d’informauons»  que  le  controlcur  construit  lui-mcme  ps  ses  propres  processus  intellectuels  et  qu’il  «balaie»  en  permanence  ps 
la  pensde. 

e)  Toute  nouvclle  visualisation  additionnelle  doit  obdir  au  double  entire  suivant : 

-  elle  doit  etre  gdrde  automadquement  ps  le  calculates, 

-  elle  ne  doit  prdsenter  aucune  discontinued  dc  visualisation  sauf  lors  de  l’affichage  de  nouvelles  informations  ayant  un  csactire 
directement  pertinent  pour  i’exdcution  du  controle  proprement  dit 

0  Afin  de  faciliter  le  dialogue  controleur/calculateur,  une  telle  visualisation  caihodique  doit  servir  de  support  pour  des  dchanges 
d'lnformation  avec  le  calculateur  (ddsignation  tactile  ou  autre) 

Pour  pouvoir  ddcrire  un  tel  projet,  il  est  ndeessaire  de  rappeler  au  prdalable  les  grandes  classes  de  taches  qui  sont  exdcutdes  ps  les 
controleurs. 

g)  Ce  projet  n’exclut  en  aucune  maniire  la  mise  en  oeuvre  d'autres  visualisations  qui  remodileraient  la  position  de  contrdle  classique 
actuelle,  mais  qui  ne  sont  pas  en  tant  que  tel  les  indispensables  i  la  mise  en  oeuvre  de  la  prdseme  thdorie. 


LE  TRAVAIL  DES  CONTRdLEURS 
Classification  des  taches 

Les  taches  dldmentaires  pour  1’exdcuuon  du  controle  peuvent  etre  recensdes  de  la  maniire  suivante  :■ 

a)  recherche  des  nouveaux  conflits. 

b)  «dvaluation»  de  ces  conflits  dans  le  cadre  de  t’ensemble  des  «conflits  en  cours»  ddji  reconnus, 

c)  ddcision  :• 

•  cl)  soil  de  rdsoudre  immddiatement  le  conflit, 
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-  c2)  soil  de  diffdrer  sa  resolution  et  d'ajoulcr  ce  nouveau  conflit  4  1'ensemble  des  «conflns  en  cours#, 

d)  surveillance  de  revolution  de  chacun  des  «con(ltts  en  cours»  en  vuc  . 

-  dl)  de  s’assurer  qu'ancune  situanon  dangcreuse  n’est  en  train  de  se  ddveloppcr, 

-  d2)  de  decider  pour  chacun  de  ces  conflits  en  cours  le  moment  optimal  pour  proceder  4  sa  resolution, 

e)  choix  de  la  solution  et  execution  de  la  resolution  des  conflits  au  moment  juge  oppottun  4  la  suite  des  processus  c  1  ou  d2, 

f)  coordination  au  moment  oppottun  avec  les  autres  secteurs  de  controle. 

Chacune  des  ces  laches  eiementaires  ressortit  4  l’une  des  quatre  classes  suivantes  :■ 

-  recherche  des  conflits  (a), 

-  suivi  des  conflits  (dl), 

-  ordonnancement  des  laches  (b,c,d2,f), 

•  execution  des  actes  de  controle  proprement  dits  (e). 

Les  taches  des  trois  premitres  classes  sont  invisibles  par  un  observateur  •  elles  se  deroulent  integralement  «dans  la  tete  du 
controleur#  et  font  partie  de  son  «savoir-faire»;  les  processus  intellectuals  sous-jacents  sont  extremement  complexes  et  done  difficiles  4 
exploiter  et  4  enseigner  a  priori. 

Recherche  des  confuts 

La  recherche  des  conflits  potentiels  reltve  en  pnneipe  de  calculs  mathdmatiques  portant  sur  I’extrapolation  des  trajectoires  puis  sur 
leur  comparaison  deux  4  deux.  Les  calculs  4  effectuer  sont  si  nombreux  et  sont  d'une  complexild  telle  qu’ils  seraient  inaccessibles  au 
cerveau  humain  travaillant  en  temps  reel  et  en  calcul  mental. 

Le  controleur  est  done  amend  4  proedder  d’une  manitre  heunsuque ;  la  pertinence  de  ses  dvaluations  et  le  caracttre  optimum  de  ses 
apprdcianons  sont  ainsi  entachds  d’un  flou  d'autant  plus  important  qu’il  ne  peut  dvaluer  que  d’une  manitre  approximative  la  valeur 
actuelle  de  certains  paramttrcs  (notamment  la  Vitesse  des  avions  et  leur  taux  devolution  dans  le  plan  vertical)  et  a  foiticn  les  aldas  qui 
affecteront  la  conduite  de  la  suite  des  vols. 

Certaines  ddclarations  de  «non  conflit»  ne  peuvent  done  pas  etre  considdrdes  comme  absolument  sures  et  ddfinmves. 

Elles  s’effectuent  implicitement,  en  ne  tenant  compte  que  d’une  panic  de  la  courbe  de  distribution  siansiique  des  erreurs  et  aldas  et 
en  ndgligeant  les  «queues  de  distnbution»  au-del4  d’une  certaine  probabilud 

Une  telle  simplification  est  mdluctable  ,  elle  impose  en  contrepanic  une  «surveillance»  de  1’dvolution  de  la  situation  d’autant  plus 
vigilante  que  les  marges  de  sdparaiion  dvaludes  s'approchent  plus  de  la  limite  admissible.  Cette  surveillance  consume  en  soi  une  t4che 
spdcifique  et  provoque  une  inquidtude  latentc ;  elle  alourdit  le  travail  du  controleur,  et  nuit  4  sa  dispombilitd  d’espnt,  un  oubli  peut 
donner  lieu  4  incident  ou  accident. 

Pour  limiter  l’ampleur  des  combinatsons  d’dvdnements  4  prendre  en  considdration,  le  calculates  devra  aussi  proedder  d’une 
manidre  similaire  en  «normalisam»  les  trajectoires  pour  ne  considdrer  dans  un  premier  stade  d’analysc  que  les  dvdncments  dont  la 
probabilud  d’occurrence  ddpasse  un  certain  seuil  et  en  dliminant  ceux  qui  pourraienl  rdsulter  de  la  conjonction  d’dvdnements  peu 
probables  (6). 

Mais,  que  la  ddclaration  de  «non  conflit#  soit  humaint  ou  automatique,  le  calculates  est  en  mesure  de  proedder  ensuite  4  la 
surveillance  pennanente  de  Involution  effective  de  la  situation  ,  lorsqu’il  est  amend  4  rdviser  un  jugement  initial,  il  peut,  en  temps 
opportun,  attirer  l’attention  du  controleur  sur  le  ddveloppement  d’une  situation  critique.  II  en  tdsultera  non  seulement  un  alldgement 
important  de  la  tache  des  controlcurs  mais  aussi  une  «sdcunsauon»  qui  lui  confdrera  une  meillcse  dispombilitd  d’espnt  pos  les  auues 
taches.  Cette  foncuon  devnut  done  concourir  de  ce  double  point  de  vue  4 1’accroissement  de  la  capawtd. 

RESOLUTION  ET  SUIVI  DES  CONFUTS 

Lorsqu'un  conflit  potentiel  est  ddteetd,  notamment  4  l’entrde  d’un  avion  dans  un  secteur,  le  controleur  peut  proedder 
immddiatement  4  sa  resolution  en  demandant  au  pilote  la  modification  de  certains  dldments  de  sa  ttajectoire  (niveau  autorisd,  Vitesse, 
route...). 

11  peut  aussi  laissc."  provisoirement  les  choses  en  l’dtat.  U  doit  alors  suivre  l’dvolution  de  la  situation ;  comme  dans  le  cas  dvoqud 
ci-dessus,  il  va  de  soi  que  le  calculates  peut  effectuer  automahquement  la  tache  de  surveillance  comespondante. 

Ordonnancement  des  taches 

On  appellera  “ordonnancement  des  taches#  l'opdration  qui  consiste  4  prendre  en  temps  rdel  les  dispositions  visant  4  ce  que 
l’enchalnement  des  taches  4  venir  se  prdsente  de  telle  manitre  qu'4  aucun  moment  dies  ne  risquent  de  saturcr  la  «capacitd»  des 
contrdleurs. 

C’est  sans  aucun  doute  la  classe  de  taches  qu .  est  la  plus  essenuelle  et  la  plus  ddlicate. 

Cette  classe  de  taches  est  effeclude  par  les  controleurs  de  manitre  heunstique ;  ses  modalitts  d 'execution  n’ont  pas  encore  fait 
l’objet  d’une  analyse  suffisamment  approfondie  pour  permettre  d’expliciter  1’ensemble  des  rtgles  qu’elles  mettent  en  oeuvre. 

11  y  a  tout  lieu  de  penser  cependant  que  chaque  conflit  nouveau  et  chaque  conflit  4  chaque  stade  de  son  dvoiution  est  affeetd 
implicitement  par  le  contrflleur  d’un  «indice»  •  que  1’on  appellera  son  «poids»  -  qui  caractdnse  ce  conflit  en  lui-meme,  ainsi  que  ce 
confiit  dans  ie  contexts  de  tons  les  autres  "Conflits  cn  cours-.  Ce  epcids#  refitte  d'une  manitre  composite : 

-  le  moment  oil  risque  de  se  produire  le  conflit  effectif. 


K-10 


■  la  somme  de  I’attenlion  rt-quise  pour  la  surveillance  des  autres  paires  d’avions  non  encore  sdpards  jusqu’au  moment  ou  le 
controleur  ddcidera  de  idsoudre  le  conflit  particul' , 

-  la  reflexion  imposde  quant  au  choix  du  moment  <•!  de  la  solution  pour  la  resolution  du  conflit, 

-  la  difficult  de  Faction  d’dvitement. 

Ce  vpoids*  doit  aussi  temr  cerrpte  du  temps  requis  pour  I’accompltssement  de  ces  laches,  ainsi  que  de  I’accroissement  de  la 
difficult  due  au  chevauchement  dans  le  temps  de  l’ensemble  des  taches  dldmentaires  impliqudes  par  l’ensemble  des  «conflits  en 
cours* ;  il  rdsulte  sunout  de  la  probabilitd  que  se  ddveloppe  une  situation  mdesirable,  par  exemple  telle  que  le  controleur  Radar  aurait 
par  la  suite  plusieurs  dvitements  Radar  4  effectuer  simultanement. 

Le  choix  optimal  du  moment  et  du  mode  de  resolution  de  chacun  des  conflits  relive  done  bien  d'un  probldme  d’ordcnnancement 
des  taches  du  controleur. 

De  surcroit,  le  probldme  est  compliqud  par  le  fait  que  dans  le  systdme  actuel,  cet  ordonnancement  est  effectud  4  la  fois  par  les  deux 
controleurs  constituant  l’dquipe  :  le  controleur  dit  «orgamque»  (ou  stratdgique,  ou  de  planificauon.  .)  et  le  controleur  dit  «tactique» 
(contrdleur  Radar). 

Cette  amculanon  interne  au  secteur  est  particuliirement  ddltcate ,  pour  etre  mende  4  bien,  elle  implique  la  coherence  d’une  dquipe 
soudde  ayant  l’habitude  de  coordonner  son  travail  d'une  mamiie  organ isdc  mats  lacite. 

II  s’agit  du  problime  crucial  dont,  4  notre  avis,  ddpend  I'dvoluuon  du  condole  .  on  peut  s’en  convaincre  cn  rappelant  que  la 
saturation  du  controle  ne  rdsulte  pas  tant  de  la  saturation  effecuve  de  1‘espace  que  de  la  saturation  par  des  «pointes  d’accumulation  de 
taches*  (et  notamment  par  la  probabilitd  d’avoir  4  mener  simultandment  deux  dvitements  Radar)  qui  ddpassent  ce  que  peut  exdcuter  en 
un  temps  donnd  chaque  contrdleur  ou  chaque  dquipe  de  condole. 

Le  bon  ordonnancement  de  ces  taches  dans  le  temps  consume  done  bien  le  problime  central  auquel  il  convient  de  s'attaquer. 

C’est  ainsi  que  la  capacitd  et  la  sdcuritd  d’un  secteur  de  condole  sont  directement  fonction  pour  chacun  des  conflits  • 

-  du  choix  du  moment  optimal  pour  entreprendre  sa  risolution, 

-  du  choix  de  la  soluhon  opnmale,  d’une  mamire  telle  que  son  minimisde  la  probabilitd  d’accumulauon  pemicieuse  de  taches  4  un 
moment  futur  quelconque  et  que  soit,  si  possible,  lissee  dans  Ic  temps  la  «demande  d’attention*  des  controleurs. 

C’est  en  priontd  l’assistance  4  cet  ordonnancement  des  taches  que  le  projet  se  propose  d’automatiser  ainsi  que  le  «filet  de 
sauvegarde*  associd,  d’oii  le  nom  du  projet  FREGATE  (Filtre  de  Rdgulation  Ergonomique  de  la  Gestion  [des  secteurs]  et 
d’ Anticipation  des  Taches  Excessives).  Ces  fonenons  nouvelles  sont  complexes  et  mdntcnt  un  examen  approfondi. 


LA  REALISATION  DU  PROJET 
LES  FONCTIONS 

Si  le  projet  implique  le  foncuonnement  en  parallile,  et  avec  un  minimum  d’inieracuon,  du  controleur  et  du  calculates,  il  n’en  reste 
pas  moms  que  ces  deux  processus  doivcnt  se  rejoindre  chaque  fois  que  le  calculates  est  susceptible  de  rendre  un  service  effecuf  aux 
controleurs. 

Il  ressort  de  l’analyse  ci-dessus  que  1’interface  entre  le  contrdleur  qui  agit  et  le  programme  parallile  de  controle  automatique  qui  est 
latent,  doit  s’effectuer  sous  deux  formes  compldmentaires  . 

-  l’aide  4  l’ordonnancement  des  taches :  AOT, 

-  l’alerte  en  cas  de  prdvision  oar  le  calculateur  du  ddveloppement  d’une  situation  future  indisirable  du  point  de  vue  de 
(’accumulation  des  taches  et  i  fortiori  d’une  situation  dangereuse  :  c’est-4-dire  un  «filet  integral  de  sauvegarde  tactique  et 
orgamque*  (F1STO) 

On  notera  que  ce  FISTO  consume  une  giniralisation  de  la  notion  acluelle  de  «filet  de  sauvegarde  Padar*  mais,  alors  que  ce  dernier 
n’intervient  que  pos  alarm er  en  cas  de  danger  4  tris  court  terme,  le  FISTO  peut  alerter  pos  prdvenir  de  toute  situation  indisirable  du 
point  de  vue  de  l’accumulation  des  tSches  qui  risque  de  survenir  4  tout  moment  dans  le  futur.  Contrairement  au  cas  du  «filet  de 
sauvegarde  Radar*  qui  impose  une  acuon  immddiate  et  rapide  du  controleur,  le  FISTO  n’est  nullement  impiratif,  (on  montrera  qu’il 
devra  etre  conqu  de  telle  maniire  qu’il  sou  de  l’intdrel  du  controleur  d’en  suivre  les  avis). 

A  cette  fin,  il  faut  que  le  ‘programme  de  contrdle  automatique  latent*  ddbouche  sur  une  visualisation  qui,  4  la  fois,  permette 
l’assistance  4  l’ordonnancement  des  taches  et  procure  le  filet  de  sauvegarde  associd.  Mais  il  convient  aussi  que  cette  visualisation  soit 
telle  que,  par  simple  resignation  tactile,  elle  puisse  irmplir  deux  Sanctions  compldmentaires  lorsqu’un  contrdleur  utilise  effectivement 
l’AOT  ou  ddcide  de  rdpondre  aux  sollicitations  du  FISTO,  4  savoir  :• 

-  une  aide  au  choix  des  solutions  (ACS), 

-  une  aide  4 1’entrde  de  la  solution  choisie  (AES). 

A  partir  de  ces  objectifs  et  de  ces  principes,  il  devient  possible  d’imaginer  une  visualisation  nouvelle  susceptible  de  servir 
facilement  d'interface  homme/calculateur  pour  1’exdcution  des  fonctions  qui  viennent  d’etre  analysdes. 

UNE  VISUALISATION  ORIGINALS 

Description  de  la  visualisation 

Les  besoins  recensds  ont  conduit  4  imaginer  une  visualisation  d’une  nature  origmale 
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Les  quatre  lonctions  AOT,  F1STO,  ACS,  AES  peuvent  s'articuler  autour  d’une  visualisation  unique  au  profit  des  deux  controleurs 
organique  et  tactique,  ou  mteux  sur  deux  visualisations  identities  placdes  respectivement  4  la  vue  et  it  la  portde  de  chacun  de  ces  deux 
controleurs. 

Bile  est  de  type  analogique  comme  les  ecrans  radar  mais  4  la  diffdrence  de  ceux-ci,  chaque  point  affichd  (et  l'dtiquette  qui  lui  cst 
attachde)  ne  porte  pas  sur  un  avion  individuel  mais  reprd sente  toujours  une  paire  d'avions  en  confiu  potentiel.  Ces  conflits  sont  classds 
en  fonction  du  moment  auquet  ils  interviendront  si  aucune  mesure  n’dtait  pnse  entre-temps. 

Cette  visualisation  se  ptdsenterait  de  la  tnanidre  suivante : 

a)  Sur  un  dcran  rectangulaire  horizontal  est  traede  une  ligne  mddiane  horizontale  ayant  une  ongine  O  et  servant  de  gauche  4  drone 
d’axe  des  temps. 

b)  Sur  cet  axe  sont  figurds  des  points  reprdsentant  les  «conflits  d’avions  deux  it  deux  non  encore  rdsolus».  Un  avion  impliqt'd  dans 
plusieurs  conflits  est  reprdsentd  par  plusieurs  points 

c)  L'dlongation  de  ces  points  sur  l’axe  des  temps  teprdsente  la  durde  qui  va  s’dcouler  entre  le  moment  actuel  et  le  moment  oil  aurait 
lieu  le  conflit  effectif  si  aucune  mesure  de  rdsoluuon  n'dtait  prise  d’ici  lit. 

d)  Chaque  point  est  affeetd  d’une  numdro  d’ordre  et  est  relid  it  une  dtiquette  par  un  tiret  de  rappel  permettant  d’identifier  I’dtiquelte 
correspondante. 

Sous  des  formes  optimales  it  expdrimenter  (clnffres,  couleurs,  symbole,  dpaisseur  de  tiret,  etc ),  le  point  et/ou  l’dtiquette  donnent 
une  indication  du  «poids»  du  conflit  et  de  son  urgence  (au  sens  ddfini  ct-dessus). 

e)  Chaque  dtiquette  comprend  l'indicatif  des  deux  avions  concemds. 

0  L’dctan  comprend  trois  parties  :• 

-  la  partie  centrale  uulisde  de  la  manidre  ddcrite  ct-dessus, 

-  une  partie  haute  dans  laquelle  le  calculates  trace  1'histogramme  de  la  sommation  4  chaque  instant  futur  de  la  somme  des  taches 
«ponddrdes»  par  les  «poids»  respectif  qui  va  ddcouler  de  la  gestion  et  de  la  rdsolution  de  tous  les  «confins  en  cours», 

-  une  partie  basse  servant  de  clavier  d'entide  tactile  (ou  autre)  adapt d  it  chaque  instant  par  le  calculates  4  la  (ache  probable  que  le 
controleur  va  etre  amend  it  effectuer 

Par  adless,  I’dtiquette  de  l’dcran  Radar  classtquc  comprendra,  outre  les  donndcs  actuelles,  le  rappel  des  numdros  des  conflits  non 
encore  rdsolus  dans  lesquels  est  impltqud  I'avton  considdrd. 

Fonctions  de  la  visualisation 

a)  AOT :  le  seul  examen  de  la  visualisation  permet  au  contrdleur  de  prdvoir  sa  charge  future  de  travail,  de  prendre  conscience  en 
temps  opportun  de  tout  risque  de  saturation,  de  ses  causes  et  des  mesures  k  prendre. 

b)  FISTO  .  si  le  controleur  n’est  pas  assez  attentif  4  certatnes  dventualuds  d’accumulation  potenticlle  des  taches,  ou  ne  s'est  pas 
montrd  en  mesure  d’opumiser  sa  charge  future  de  travail,  le  calculates : 

-  ddclenchera  une  alerte  4  partir  de  certains  seutls  de  1’histogramme  des  taches  futures; 

ddsignera  le  (ou  les  conflits)  qu’il  estime  appropnd  de  rdsoudre  dfcs  mamtenant.  Cette  ddstgnatton  s’cffectuera  par  exemple  en 
faisant  «flasher»  le  point  reprdsentatif  du  conflit  sur  I'dcran  ddent  ct-dessus  (ainst  que  les  pistes  correspondantes  sur  1'dcran 
Radar). 

On  rappelle  que  la  prdsentation  de  ces  suggestions  n’tmplique  pas  obligatoirement  une  action  du  controleur  (qui  reste  fibre  de  sa 
strategic)  mais  constuuera  sans  aucun  doute  pour  lui  une  aide  prdcieuse.  Toutefois,  lorsque  le  calculates  esnmera  que  la  situation 
risque  de  se  transformer  en  une  situation  dangereuse  4  terme,  1’alerte  pourra  se  transformer  en  alatme  et  devenir  imperative 

Ce  nouveau  dispositif  propose  n’est  pas  limite  aux  quelques  secondes  qui  precedent  un  conflit  effectif  comme  dans  le  cas  du  «nlet 
de  sauvegarde  Radar»  mais  s'dtend  en  un  continuum  4  toute  periode  precedant  un  confiit  potentiel  donne.  II  constituera  un  outil 
fondamental  du  contrdle  proprement  diL 

On  notera  aussi  que  cctte  extension  de  la  notion  de  «filet  de  sauvegarde»  permet  de  protdger  le  controleur  contre  route  faute  de 
jugement  grave  et  notamment  contre  tout  oubli  d'une  situation  conflictuelle  precedemment  ddtectde,  mais  occultde  par  1’execution 
d’autres  tSches  qui  retiennent  toute  son  attention. 


c)  ACS :  lorsque  le  controles  ddcide  d’agir  pour  rdsoudre  un  conflit  donnd,  soit  de  sa  propre  initiative,  soit  parce  qu’il  reconnatt  le 
bien-fondd  d’une  alerte  ou  d'une  alarme  FISTO,  il  lui  suffira  de  ddsigner  l'dtiquette  du  conflit  considdrd. 

Le  calculates  fera  alors  apparaitre  dans  la  partie  basse  de  la  visualisation  un  choix  de  solutions  par  ordre  de  prdfdrence,  c’est-4-dire 
que  les  solutions  seront  classdes  d’une  manidre  d'autant  plus  prioritaire  que  leur  mise  en  auvre  lisse  mieux  la  tache  globale  moyenne 
future  du  contrdleur  ou,  en  cas  de  besoin,  minimise  les  pointes  futures  d’accumulation  de  ces  taches.  a 


Toutes  les  solutions  raisonnables  (changements  ou  limitations  de  niveau,  changement  de  route,  changement  de  vitesse 
iongitudinale  ou  verticale,  dvitemem  Radar  immddiat  ou  diffdrd,  action  sur  un  des  avions  ou  sur  1’autxe...)  peuvent  etre  affichdes ; 
certaines  d’entre  elles  devront  cependant  etre  affeetdes  d'un  symbole  d’interdiction  en  raison  des  nouveaux  conflits  qu’elles  sont 
susceptibles  d'induire  avec  d’autres  avions,  ou  du  fail  qu’elles  sont  de  nature  4  ddstabiliser  I’organisation  du  trafic  telle  qu’elle  a  dtd 
preeddemment  prdvue.  Parmi  ces  solutions  pourra  figurer,  dds  que  cela  sera  techniquement  possible  (mode  S  +  FMS),  la  pnse  en 
comptc  dirccte  dc  calculateur  Sol  &  calculateur  Air  du  guidage  des  avions. 


Si  le  contrdleur  choisit  I’dvitement  Radar  diffdrd,  le  calculateur  surveillera  la  patre  d’avions  considdrde  et  rappellera  au  contrdleur 
sous  lorme  d  alarme  ce  conflit  non  rdsolu,  suffisamment  4  l’avance  pour  lui  permettre  d’assurer  en  toute  sdrdnnd  I’dvitement  Radar.  Le 
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controleur  est  ainsi  placi  4  1‘abri  de  la  craime  d'oublier  un  des  conflus  Radar,  au  cas  ou  son  attention  et  ses  capacitis  seraient  saturics 
par  1  ’execution  d’autres  laches. 

L’ordre  de  prifirence  du  classement,  de  meme  que  les  solutions  «mterdttes»  peuvent  temr  compte  de  la  prisence  d’avions  ivoluant 
actuellement  dans  un  autre  secteur,  c’est-4-dtre  que  l'optimisation  de  l’ordonnancement  et  du  choix  des  solutions  couvre  un  domatne 
plus  vaste  que  celui  auquel  s’intiressent  les  controleurs  d'un  secteur  considiri . 

C’est  ainsi  que,  de  proche  en  proche,  la  notion  de  «secteur  de  contro!e»  pourra  perdre  sa  ngiditi  actuelle  et  que  le  processus  de 
segmentation  croissante  de  I’espace,  seule  solution  acntelle  (aux  rendements  d’ailleurs  dicroissants)  it  1’augmentatton  de  la  capacite  de 
l’espace,  pourra  etre  lenversi  ct,  en  tout  cas,  nuira  de  moins  en  moms  it  l'optimisation  de  l’unltsation  globale  de  l’espace. 

d)  AES  :  Lorsqu’un  controleur  dticide  d’agtr  sur  un  conflit,  il  dtsposera  ainsi  d’un  choix  de  solutions  II  lut  suffira  de  designer  du 
doigt  la  solution  stilectionnie  (en  pnncipe,  mats  non  obligatotrement  celle  qut  est  prisentie  la  premiire  par  le  calculateur).  Cette 
action  aura  pour  effet  d’informer  le  calculateur  du  choix  du  controleur  (el  ceci  est  essentiel  pour  assurer  la  stability  de  tout  le 
processus  automatique). 

Mais  cette  action  tris  simple  pourrait  avoir  de  surcroit  pour  effet  :■ 

-  de  faire  envoyer  ces  ordres  par  liaison  automatique  de  donnies  (quand  le  systime  sera  arrive  4  ce  stade  de  developpcmcnt)  puts 
d’assurer,  par  la  suite,  plus  ou  moms  complitement  et  automatiquement,  des  manoeuvres  complexes  par  cotiplage  du  calculateur 
sol  et  des  FMS  des  avions ; 

•  de  faire  envoyer  iventuellement  les  ordres  correspondants  par  voix  artificielle  aux  avions  non  iquipis  de  liaisons  codies 
automatiques; 

-  d’informer  automatiquement  le  systime  militaire  assoctd  des  autorisations  correspondantes,  rdsolvant  ainsi  un  problime 
lancinant  et  essentiel  pour  l’efficaciti  de  la  coordination  ctvtle/militaire. 

e)  ACI .  la  visualisation  peut  enfin  servir  d’aide  aux  processus  de  coordination  intersecteurs  et  assurer  leur  «survetllance»,  il  savoir 

-  qu’elle  peut  servir  de  support  intelligent  it  la  coordination  mtersecteur  en  ce  sens  que  toute  pnse  de  ddciston  influant  sur  les 
conditions  de  transfert  intersecteur  peut  faire  I'objet  d'un  message  au  secteur  suivant  (si  le  controleur  sun  les  propositions  du 
calculateur,  les  conditions  de  transfert  qui  out  ili  dlabordes  par  ce  dernier  de  rnamire  telle  qu’elles  soient  acceptables  par  le 
secteur  suivant); 

-  qu’elle  peut  servir  de  «filet  de  sauvegarde  de  coordination))  en  atnrant  l'aiteniion  du  controleur  sur  lout  avion  s’appiochant  des 
ltmites  du  secteur  et  n’ayant  pas  encore  fait  I’objet  d’une  coordination  avcc  le  secteur  suivant. 

0  En  conclusion,  on  von  que  la  vtsua'isation  proposie  consiitue  4  la  fois  une  aide  dirccte  au  controleur  pour  l’ordonnancement  de  ses 
actions  (dont  il  garde  une  complete  maitnse)  amsi  qu’une  protection  efficace  contre  tout  oubli  ou  distraction. 

On  peut  espirer  que  le  potentiel  de  l’dquipe  de  contrfile  sera  largement  acctu  par  le  «calcu!ateur  controleur  vutuel  automauque»  (le 
troistime  homme  fictif)  4  1’image  des  ptlotes  qui  peuvent  se  concentrer  sur  les  laches  qu’ils  sont  les  plus  apres  4  effectuer  et  garder 
routes  leurs  potentiality  4  leur  optimum,  lorsqu'ils  sont  dichargis  de  toutes  les  laches  de  surveillance  et  d’exticutton,  que  les  pilotes 
automatiques  (voire  le  FMS)  peuvent  effectuer  sans  leur  concours. 

Cette  visualisation  seratt  aussi  un  instrument  idial  de  coordination  entre  le  controleur  «orgamque»  et  le  controleur  Radar  •  elle 
consutue  en  ce  sens  une  aide  pr&ieuse  4  la  coordination  intrasecteur. 

Cette  visualisation  serait  aussi  particuliirement  pidcieuse  au  moment  de  la  relive  d’une  iquipe  par  une  autre  lors  des  changements 
de  quart,  elle  synthiuse  en  effet  4  tout  instant  la  vie  actuelle  et  future  du  secteur.  C’cst  dire  l’ef  ficaciti  qu’on  est  en  droit  d’en  espirer 
en  situation  normale,  et  i’dtat  de  ddtente  des  controleurs  qu’on  peut  en  escompier  ceux-ct  seront  non  seulemem  assistis  dans 
1’ordonnancement  de  leurs  taches,  mats  pourront  se  consacrer  sans  apprihension  4  la  conduite  d’une  seule  tSche  4  la  fois  (notamment 
la  conduite  d'un  ivitement  Radar)  sans  etre  agressis,  d’une  mamire  permanente  et  latente,  par  la  craime  d’oubher  un  autre  conflit  ou 
d’avotr  laissi  se  ddvelopper  une  situation  qut  pourrait  devemr  difficile  4  maitnser  par  la  suite. 


LA  DIFFICULTE  CONCEPTUELLE 

On  peut  se  demander  dans  quelle  mesure  ce  projet,  dont  les  lignes  directrices  ont  titi  dicrites  ci-dcssus,  pourrait  se  riviler  de 
nature  4  permettre  de  surmomer  la  difficult  conceptuelle  fondamentale  qui  s'oppose  au  franchissement  du  «mur  de  la  capacity. 

Le  calculateur  ne  peut  tenir  les  visualisations  4  jour  et  ne  peut  proposer  et  classer  les  soluuons  que  s’il  est  tenu  informi  de  toutes  les 
decisions  effectivement  retenues  de  proche  en  proche  par  les  controleurs. 

Mais  il  doit  aussi  pouvoir  faire  des  hypothises  rdalistcs  sur  les  decisions  que  le  controleur  sera  amend  4  prendre  dans  le  futur.  Pour 
pouvoir  riduire  le  champ  du  probable  voire  du  possible,  le  calculateur  devra  lui-memc  connaltre  les  origles  de  l’art»  du  controle  (les 
controleurs  ne  ddcident  pas  n’impoite  quoil). 

A  cette  fin,  il  faudra  programmer  le  calculateur  4  l'image  de  I’expertise  des  controleurs  et  done  expliciter  au  prdalable  avec  I'aide 
des  controleurs  eux-mimes,  1’ensemble  complexe  des  rigles  qu'tls  appliquent  dans  chacune  des  multiples  situations  qui  nsquent  de  se 
presenter,  et  des  potds  d’angoisse  qu’tls  attribuent  4  l'apprihension  des  tiches  4  venir. 

Il  y  a  tout  lieu  de  penser  que  le  ddveloppement  de  ces  «programmes  expens*  son  possible,  mais  on  sail  dij4  qu’ils  comprendrom 
un  nombre  considdrable  de  cas  et  de  «rigles»  (plusieurs  milliers)  et  qu’ils  ne  pourront  etre  perfectionnds  que  par  un  long  processus 
auquel  les  controleurs  doivent  etre  intimement  associes. 

Les  techniques  dites  de  I’intelligence  artificielle  et  les  mithodes  modemes  d’explicitation  des  rigles  que  les  controleurs  appliquent 
sans  en  avoir  une  conscience  explicite,  ouvrent  des  voies  iris  encouragcantes  comme  le  montrent  en  particular  les  premiers  risultats 
des  travaux  de  Leroux  4 1’ENAC. 


LE  PASSAGE  DU  MUR  DE  LA  CAPACITY  :  LES  BOUCLCS  VERTUEUSES 


Pius  el  mieux  ie  calculates  influencera  les  choix  des  contrdleurs,  plus  le  «charop  de  l’ir, certain®  pourra  etre  rdduit  (l’expirience 
seule  pourra  montrer  dans  quelle  mesure  les  choix  prifdrenticla  du  calculates  seront  cffectivemenl  retenus  par  les  contrdleurs)  et,  en 
consequence,  plus  les  choix  proposes  seront  realistes  et  plus  le  systime  sera  stable.  Plus  les  contrdleurs  estimeront  pouvoir  st  laisser 
influencer  par  le  calculates,  meilleure  sera  l’efficacite  du  calculates  et  plus  granae  la  satisfaction  des  eontrdleurs. 

Le  projet  recile  ainst  des  «bouc!es  vertueuses®  qut  devratent  facih'er  son  introduction  et  penntttre  sa  montee  cn  puissance  :  tout 
accroissement  d’efficacite  eiementaire  acaoit  le  service  rendu,  qui  lui-mime  tend  k  son  tour  ii  aceroltre  la  faisabilitr'.  et  Tefficaciti  du 
projet  dans  son  ensemble. 

11  en  est  de  meme  de  l’lnformation  du  calculates  des  decisions  effectivement  retenues  par  les  contrdleurs:  les  controieurs  auront 
tnliret  k  faire  connaitrc  leurs  decisions  au  calculates  pour  rendre  celui-ci  plus  efficace  dans  1’ensemble  dee  services  qu’il  lui  rend. 
C’est  ainsi  que,  plus  le  calculates  sera  efficace  dans  I’ilaborauon  de  choix,  plus  ii  y  a  de  chance  que  ces  choix  soieot  suivis  par  le 
contrdles. 

Or  l’entrie  dans  le  calculates  de  la  solution  choisie  est  con$ue  pour  se  faire  d’une  maniire  particuliiremcnt  aisde  pou.  les 
controieurs,  les  contribution  se  boma  it  k  la  designation  de  la  solution  retenue  panni  les  choix  proposes.  Lorsqu’ils  estimeront 
pouvoir  ainsi  informer  le  calculates  de  less  decisions,  les  controieurs  seront  largement  graufiis : 

a)  amelioration  de  la  pertinence  des  solutions  proposecs  par  le  calculates, 

b)  suivi  auiomatique  des  autorisations  de  controle, 

c)  surveillance  automatique  des  conflits,  alettes  et  alarmes  F1STO, 

d)  coordination  automadque  intra-  et  inter-sectess, 

e)  information  des  militants,  etc. 

U  existe  done  dans  le  systime  plusieurs  «boucles  vertueuses»  qui  devraient  faciliter  son  introduction  et  son  amelioration  de  maniire 
iterative  et  qui  feront  que  •• 

-  plus  les  controless  se  serviront  du  systime,  plus  lls  seront  directement  gratifies,  done  plus  le  systime  sera  efficace  pour  les 
controieurs, 

-  plus  le  systime  sera  efficace  pour  les  controieurs,  plus  ils  s'en  serviront. 

La  boucle  a  des  chances  de  se  boucler !  Tel  est  le  pan  du  projet. 

11  convient  de  noter  que,  si  le  processus  d'analyse  de  la  situation  par  le  calculates  est  «&  !’ image®  de  celle  qu'effectue  le 
conttdleur,  le  calculateur  disposera  de  moyens  plus  puissants  que  ceux  du  conttdleur  pour  les  traiter  exhaustivement,  ainsi  que  sur  une 
zone  dipassant  celle  d'un  seul  secteur  individuel. 

De  son  cotd,  la  «surveillance»  de  la  situation  et  notammenl  des  conflits  non  encore  rdsolus  constitue  un  processus  purement  logico- 
mathdmatique  exempt  du  flou  heunstique  des  processus  de  ddcision. 


UN  PROJET  MOBIL ISATEUR  POUR  LES  ANNIES  90 

Ce  projet  consume  le  prolongement  pratique  des  objectifs  et  de  la  theone  esquissds  dans  I’anncxe  7  du  rapport  des  Sages  (7).  II  ne 
devrait  pas  rencontrer  d'obstacles  majeurs  de  nature  psychologique  ou  sociale. 

Si  son  processus  ltdratif  tenait  ses  promesses,  il  constituerait  une  voie  possible  d'augmemation  de  la  capacttti  de  1’espace  (dont  on 
salt  qu'elle  est  essenuelletnent  limitde  par  l'accumulation  des  laches  plutot  que  par  le  manque  d’espace  proprement  dil),  de  dimmunon 
des  stress  des  contrdleurs  et  ccrrdlativement  d'augmentation  de  la  sdcuritd  globrle 

II  prdsente  le  mdrite  de  permettre  de  se  greffer  sur  le  systime  actuel  et  notamment  son  logicicl  et  ses  sorties  (strip  papier  ou 
cathodique,  visualisation  Radar  et  digitatron).  II  pourrail  bdndficier  cependant  du  remodelage  de  la  position  de  controle  par  des 
visualisations  plus  intelligentes  (cf.  PHIDIAS  par  exemple). 

II  ne  nicessite  pas  impirativement  de  liaisons  de  donnies  aulomatiques  air/sol  pour  son  introduction ;  il  se  justifie  en  lui-mcme ,, 
mais  le  projet  une  fois  implanti,  permettrait  de  tirer  tout  le  profit  de  cette  novation  technologique,  notamment  le  couplage  automatique 
aux  FMS. 

Cependant,  ce  projet,  malgri  son  tris  grand  dipouillement  et  sa  simplicity  apparente  (notamment  en  ce  qui  conceme  1'addition  de  la 
seule  nouvelle  visualisation),  exige  un  logiciel  aussi  important  et  complexe  qu'un  logiciel  qui  assurerait  le  controle  enttirement 
automauque. 

Il  suppose  de  surcroit  non  seulement  une  connaissance  des  inches  de  controle  dans  leur  aspect  thionque,  mars  encore  une  expertise 
approfondie  des  rigles  de  1’art  et  du  comportement  effectif  des  contrdleurs  face  k  toutes  les  situations  ilimentaires  (et  aux 
combinaisons  de  situations)  susceptibles  de  se  presenter;  les  programmes  experts  5  diveloppcr  touchent  notamment  k  1’ivaluation  des 
«poids»  des  diffirents  types  de  conflits  ct  des  solutions  envisageables. 

Un  tel  projet  ne  peut  done  aboutir  que  sous  forme  de  projet  mettant  en  ceuvre  toutes  les  composantes  du  controle  :  inginieurs, 
contrdleurs,  ilectroniciens ;  en  ce  sens  il  est  mobilisateur  et  gfnirateur  de  coherence  ct,  on  l’espire,  d’enthousiasme  tnteme.  Il  devrait 
ist  conduit  commc  un  «grand  projet®. 


La  repartition  des  tSches  dldmentaires  doit  cependam  etre  suffisamment  souple  pour  ne  pas  neutraliser  la  erdauvitd  cssentielle  d  la 
maitrise  d'un  domaine  encore  largement  domind  par  I’incertain. 

n  va  de  sol,  comme  I’expdrience  passde  l'a  roujours  confirmd,  que  les  dtudes  experts  qui  seront  mendes  dans  le  cadre  de  ce  projet 
prdstnteront  un  imdrrt  considdrable  pour  batir  parallilement  un  instrument  d’enseignement  programmd  (notamment  d’enseignemem  de 
l’ordonnancement  des  tlches  et  d’dlaboranon  et  de  choix  de  solutions),  instrument  qui  devrait  permettre  des  progris  substantiels  dans 
les  durdes  et  les  qualitds  de  l’enseignement  ainsi  que  dans  l'adaptation  des  controleurs  aussi  bien  au  systime  actuel  qu’au  nouveau 
systdme.  Les  contriileurs  seraient  ainsi  instruits  sur  les  mdmes  bases  que  celtes  qui  prdsideront  it  1'dlaboranon  du  systdme  de  l’avenir. 

La  formation  et  les  dtudes  pour  le  ddveloppement  du  systdme  indent  de  pair  et  se  valoriseraient  muiuellement  L’uniftcation  it  terme 
du  systdme  europden  y  trouverait  son  compte.  On  peut  espdrer  que  les  dtudes  et  recherches  fdddrdes  par  Eurocontrol  et  par  la  CEE  et 
qui  se  ddveloppent  d’une  manihe  tres  sattsfaisante  au  sein  des  Etats  europdens,  ddboucheront  en  temps  opportun  sur  la  conception  du 
syst&me  commun  du  ddbut  du  siicle  prochain. 

L'auteur  souhaite  que  les  analyses  qui  prdeddent  puissent  apporter  leur  contribution  it  la  ddfinidon  d'un  fil  conducteur  commun  a 
ces  travaux  et  que  ses  propositions  puissent  dtre  considdrdes  comme  une  des  formes  possibles  de  leur  abounssement. 

De  son  cotd,  l'Ecole  nauonale  de  l'aviation  civile  (ENAC)  a  ddjh  re{u  un  contrat  du  Centre  d’dtude  de  la  navigation  adrienne 
franpais  (CENA)  pour  ddvelopper  sous  forme  de  maquettes  probatoires  certains  dldments  essentiels  de  ce  projet. 

La  tachc  sera  longue  et  ardue,  mais  les  enjeux  sont  essentiels. 


NOTES 

(1)  Cf.  «Pour  une  politique  europdenne  du  transport  adnen».  J.  Villiers  in  IT  A  Magazine  n°  56  et  57. 

(2)  Voir  «L’intelligence  artificielle  dans  le  contrfile  de  la  circulation  adnenne».  J.  ViUiers  in  IT  A  Magazine  n°  43,  mai/juin  1987. 

(3)  Ce  ndologisme  un  peu  scabreux  dvoque  les  chameaux  dont  on  dn  qu’ils  ne  seraient  qu’une  version  de  cheval...  conijue  par  un 
Comitd  1 

(4)  Pour  la  notion  de  «filtres»  successifs,  voir  «La  Mdthode  des  filtreso,  J.  Villiers  in  Revue  de  I’lnstitut  de  Navigation,  n°  61,  ler 
mmestre  1968. 

(5)  11  va  de  soi  que  le  «Ubre  arbnreo  du  calculates  ne  rdsulte  que  de  celui  du  conceptcs  de  son  logiciel. 

(1)  Cf.  «La  mdthode  des  filtres*  ui  Revue  de  Tlnsiitut  de  Navigation,  n°  61  (1968),  ddjh  citde. 

(2)  Rapport  de  la  «Cotnmission  de  rdflexion  sur  le  service  public  de  la  navigation  adnenne»,  janvier  1985. 
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The  Pilot’s  Associate  program  has  provided  a  series  of  technology  demonstrations  of  the  po¬ 
tential  of  integrating  intelligent  systems  and  artificial  intelligence  technology  into  modern  avion¬ 
ics  systems.  The  Defense  Advanced  Research  Projects  Agency  and  the  United  States  Air  Force 
have  provided  funding  and  program  management  to  determine  the  potential  increases  in  mission 
effectiveness  from  such  a  system  The  Pilot’s  Associate  effort  pursued  by  Lockheed  and  its  part¬ 
ners  has  produced  not  only  prototypes  for  advancea  systems,  hut  also  new  insights  into  the  na¬ 
ture  of  the  systems  themselves  as  well  as  new  approaches  for  quickly  producing  software  for 
these  systems  The  rapid  prototyping  methods  that  have  been  utilited  have  also  provided  the  ul¬ 
timate  consumers  -  the  pilots  -  with  significant  i  vareness  of  the  operation  of  the  Pilot  s 
Associate,  and  with  many  opportunities  to  improve  the  requirements  for  such  a  system.  7  his 
paper  describes  the  evolution  of  Lockheed's  Pilot’s  Associate  System  approach  leading  to  the 
current  system  configuration  Also  described  are  some  lessons  learned  from  managing  a  large 
software  development  team  assembled  to  produce  an  unprecedented  system 

Introduction 

A  steadily  increasing  emphasis  on  the  military  pilot’s  role  as  tactician  is  a  consistent  theme 
in  the  evolution  of  Western  tactical  aviation.  This  challenging  role  for  these  human  combatants 
is  being  made  even  more  difficult  by  extensions  in  the  range  and  sear-li  volume  of  on-board 
sensors  coupled  with  dramatically  enhanced  weapons  ard  aircraft  performance.  The  new  aircraft 
system  capabilities  have  provided  individual  pilots  with  the  means  and  attendant  responsibility  to 
exercise  effective  control  and  pursue  important  tactical  goals  across  a  large  portion  of  any 
potential  aenal  battlefield  Yet  these  same  advances  in  weapons  system  capabilities,  matched  to 
seme  degree  by  improvements  in  air  and  ground  based  enemy  systems,  create  extraordinary 
demands  on  the  skills  of  aircraft  pilots  as  tacticians  and  threaten  to  overwhelm  aircrews  in  the 
future. 

The  problem  of  coping  with  these  demands  on  the  pilot's  powers  of  observation,  planning 
and  decision  making,  frequently  summarized  ir.  the  catchphrase  “situation  awareness”,  has  long 
been  recognized  by  the  operational  community  as  a  central  issue  in  assuring  combat  effective¬ 
ness.  Yet  the  essential  nature  of  this  complex,  context-dependent  question  has  confounded  ef¬ 
forts  to  complete  a  satisfactory  analysis  of  the  picbi  im.  Some  early  efforts  to  resolve  the  prob¬ 
lem  with  avionic  systems,  displays  and  software  and  hardware  enhancements  to  sensors  have  re¬ 
sulted  in  only  marginal  improvements,  or  may  have  actually  worsened  the  situation.  In  1986,  the 
U.S.  Defense  Advanced  Research  Projects  Agency  (DARPA)  initiated  the  Pilot's  Associate  pro¬ 
gram  to  investigate  an  artificial  intelligence  approach  to  the  comprehensive  problem  ’  . 


A  Description  of  the  DARPA  Riot’s  Associate  Program 

The  Riot’s  Associate  program  is  one  of  several  technology  application  efforts  in  DARPA's 
Strategic  Compuung  Program.  The  technical  goal  of  the  Strategic  Computing  Program  is  the  sic- 
velopment  of  advanced  computing  technology,  including  intelligent  systems,  new  processor  ar¬ 
chitectures  and  speech  and  vision  capabilities,  to  address  critical  military  problems.  The  Pilot’s 
Associate  program  was  created  to  apply  the  technologies  of  real-time,  cooperating  knowlcdge- 
based  systems  for  exploring  the  potential  of  artificial  intelligence  to  improve  mission  effecuve- 

ness  and  survivability  of  advanced  fighter  aircraft.1'^ 


The  Riot’s  Associate  program  is  planned  as  a  five-year,  two  phase  effort,  administered  for 
DARPA  by  the  United  States  Air  Force  Wright  Research  and  Development  Center  (WRDC). 
Following  a  pre-contract  feasibility  phase  and  demonstration  (Demo  1)  with  numerous 
contractors,  die  first  three-year  contracted  program  phase  called  for  two  non-real-time 
demonstrauons  (Demo  2  and  Demo  3)  of  a  limited  functionality  Pilot’s  Associate  system  in  a 
laboratory  development  environment.  The  second  two-year  phase  will  culminate  in  a  system 
evaluation  and  demonstration  (Demo  4)  of  a  full-function,  multi-mission  Pilot’s  Associate  in  a 
real-time  full  mission  simulation  environment.  Two  independent  development  teams,  being  lead 
by  Lockheed  Aeronautical  Systems  Company  and  McDonnell  Aircraft  Company,  have  ap¬ 


proached  the  implementation  of  the  Pilot's  Associate  from  different  directions, 
describes  the  experience  of  the  Lockheed  team. 


This  paper 


In  March  1988,  Lockheed  conducted  a  very  successful  Demo  2  of  a  prototype  Pilot’s 
Associate  that  met  or  exceeded  expectations  for  the  system  at  that  stage  of  development.  The 
system  at  that  time  was  narrowly  focused  on  an  air-to-air  mission  scenario;  it  ran  at  approximate¬ 
ly  six  times  real-time  (a  thirty  minute  mission  was  completed  in  roughly  three  actual  hours  of 
simulator  time).  This  demonstration,  viewed  by  over  200  pilots,  scientists  and  engineers,  was 
hosted  in  a  laboratory  cockpit  simulation  driven  by  one  VAX  and  six  Symbolics  computers.  It 
featured  a  real-time  replay  of  a  simulated  two-ship  air  intercept  mission  against  an  enemy  force 
of  fighters  and  bombers,  and  a  live  laboratory  run  of  the  same  mission  to  allow  interaction  and 
“what  if..."  speculations  and  system  responses.  Among  other  results,  the  demonstration  con- 
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firmed  the  feasibility  of  producing  cooperating  knowledge-based  avionic  systems,  ana  indicated 
that  such  systems  can  be  engineered  to  run  in  real-time,  and  illustrated  the  operational  utility  of  a 
Pilot’s  Associate.* 

The  second  major  demonstration  of  the  first  phase,  Demo  3,  was  held  in  November  1989.  It 
featured  an  enhanced  prototype  version  of  the  Pilot’s  Associate  performing  in  a  new  and  more 
demanding  “force  protection-escort"  mission,  and  a  prototype  version  of  the  complementary 
Mission  Support  Tool  for  pre-mission  planning.  Demo  3,  which  was  similar  in  format  to  Demo 
2,  demonstrated  a  maturing  intelligent  system  with  enhanced  functionality.  This  faster  prototype 
Pilot’s  Associate  ran  at  approximately  three  times  real-time  and  continued  to  foreshadow  signifi¬ 
cant  operational  value  for  the  eventual  airborne  system.  The  success  of  this  demonstration  mou- 
vated  continued  funding  and  support  for  the  second  phase  of  the  program  which  is  currently  in 
progress.  Phase  two  will  result  in  a  more  robust,  real-time  prototype  Pilot’s  Associate  in  early 
1992.  It  includes  a  major  demonstration,  Demo  4,  at  the  end  of  the  phase,  and  an  operational 
evaluation  of  die  system  in  a  full  mission  simulation  facility  by  a  cadre  of  current  tactical  pilots. 


Figure  1:  The  Pilot’s  Associate  Program 


Lockheed’s  Pilot  Associate 

To  date,  the  principal  effort  l  as  been  focused  on  the  design  and  development  of  a  series  of 
evolving  software  prototype'  of  the  Pilot's  Associate  which  could  eventually  lead  to  a  ficldable 
avionics  system.  In  phase  one,  development  was  accomplished  in  a  variety  of  LISP  environ¬ 
ments  on  Symbolics  machines.  In  the  currem  phase,  the  system  is  being  developed  in  C++  and 
will  run  on  specialized,  advanced  “aviomcs-like”  hardware. 

Lockheed  has  divided  the  Pilot’s  Associate  system  into  five  subsystems  that  roughly  corre¬ 
spond  to  a  top-level  functional  partitioning  of  the  tactical  air  combat  domain.  This  somewhat  ar¬ 
bitrary  but  pragmauc  division  of  responsibilities  resulted  in  a  loosely-coupled  heterogeneous  sys¬ 
tem  in  which  each  subsytem  was  distinctly  implemented  according  to  an  approach  considered 
appropriate  to  the  particular  subdomain. 

A  key  aspect  of  this  configurauon  is  the  adoption  by  the  Lockheed  team  of  a  common  plan¬ 
ning  language  known  as  the  Plan  and  Goal  Dictionary.  This  language,  which  originated  as  a  di- 
rected  graph  used  in  system  design,  allows  the  rapid  promulgation  of  planning  information,  often 
in  different  formats,  to  multiple  subsystems.  Initially,  a  sixth  subsystem,  called  the  Mission 
Manager,  existed  to  provide  a  global  blackboard  as  a  central  repository  for  active  plans  and 
goals.  As  the  Pilot’s  Associate  matured,  these  ''unctions  were  decentralized  into  the  various  sub¬ 
systems  and  eventually,  the  mission  manager  subsystem  disappeared  entirely. 

The  five  remaining  subsystems  are:  the  Pilot- Vehicle  Interface,  Situation  Assessment, 
Systems  Status,  Tactics  Planner,  and  Mission  Planner.  Two  of  the  subsystems,  Situation 
Assessment  and  Systems  Status,  fall  into  the  functional  category  of  "assessment”,  and  two  oth¬ 
ers,  Tactics  Planner,  and  Mission  Planner  may  be  categorized  as  “planning”  while  the  fifth, 
Pilot-Vehicle  Interface  provides  the  hum  an -centered  focus  necessary  to  support  the  pilot’s  deci¬ 
sion-making  requirements  through  an  intelligent,  intuitive  interface. 

Figure  2  illustrates  this  allocation  of  functions.  The  figure  shows  data  coming  to  the  system 
in  the  form  of  briefed  information  about  the  mission,  threats  and  planned  tactics,  sensor  data,  air¬ 
craft  state  and  status  data,  and  inputs  from  the  pilot's  control  devices.  T  he  output  includes  com- 


mand  and  display  control  data  to  aircraft  systems  and  information  to  be  exported  to  cooperative 
platforms.  The  Riot’s  Associate  is  designed  to  both  filter  the  flow  of  this  data  to  display  only 
data  relevant  to  the  current  mission,  or  required  by  the  pilot,  and,  to  add  value  to  the  data.  The 
system  function,  then,  is  to  reason  about  and  report  what  the  data  means  with  respect  to  the  mis¬ 
sion  plan,  and  to  suggest  and  specialize  response  plans  for  situations  as  they  arise. 


Figure  2:  Pilot's  Associate  Concept 


Provided  below  are  brief  descriptions  of  the  five  major  subsystems  of  the  Lockheed  team’s 
Pilot’s  Associate. 

Pilot- Vehicle  Interface 

This  subsystem  is  responsible  for  explicit  and  implicit  communication  with  the  pilot  and  in¬ 
terface  to  the  cockpit.  The  design  integrates  interface  management,  an  adaptive  aider  and  an 
error  monitor  to  accomplish  both  assessment  and  planning  functions  It  evaluates  pilot  activities 
and  mission  events  in  order  to  interpret  and  "understand"  the  pilot’s  actions.  The  subsystem  then 
responds  by  configuring  displays  and  control  devices,  evaluating  workload  and  performance,  de¬ 
tecting  errors,  and  adaptively  aiding  the  pilot  in  the  execution  of  his  tasks  to  optimize  his  situa¬ 
tion  awareness  and  allow  him  to  focus  his  attention  on  important  or  critical  events  Finally,  the 
system  provides  a  display  that  meets  the  expectations  of  the  pilot  in  specific  situations  and  that  is 
sensitive  to  his  personal  preferences  and  techniques. 

Pilot  intent  determination  is  an  essential  function  of  the  Pilot-Vehicle  Interface  subsystem. 

By  monitoring  pilot  actions  as  well  as  the  mission  context,  the  system  is  able  to  compare  pilot 
actions  or  specific  commands  to  a  model  of  the  pilot's  plans  and  goals  The  subsystem  can  then 
determine,  without  tedious  or  distracting  explicit  input,  the  intennons  of  the  pilot  that  will  guide 
the  behavior  of  the  remainder  of  the  Pilot’s  Associate  subsystems. 

System  Status 

This  subsystem  monitors  and  analyzes  onboard  systems  to  determine  the  current  aircraft 
state  and  to  evaluate  current  system  capabilities.  Any  detected  malfunctions  anywhere  in  the  air¬ 
craft  are  evaluated  to  determine  the  degree  of  degradation  of  the  overall  system  capability  and 
assess  the  impact  of  the  degradation  with  respect  to  current  and  proposed  mission  plans.  Where 
possible,  the  system  generates  remedial  plans  and  coordinates  them  with  plan  generators  in  other 
subsystems.  All  Pilot’s  Associate  functions  are  sensitive  to  and  focused  by  the  current  mission 
state.  Thus,  for  example,  a  minor  malfunction  in  a  redundant  system  will  not  command  inappro¬ 
priate  resources  in  the  heat  of  battle,  but,  plans  that  require  the  use  of  some  degraded  system  at  a 
later  time  in  the  mission  will  be  constrained,  adjusted  or  not  proposed. 

Situation  Assessment 

The  Situation  Assessment  subsystem  monitors  events  external  to  the  aircraft  to  provide  com¬ 
bat  situation  analysis.  It  combines  stored  mission  data  with  data  from  the  sensor  suite  and  coop¬ 
erative  sources  to  provide  context-sensitive  information  to  the  pilot  and  mission  state  information 
to  other  subsystems.  For  example,  the  target  value  can  be  assessed  with  respect  to  both  mission 
objectives  and  ownship  status.  Also,  threat  data  are  assessed  based  on  "inferred”  lethality  and 
intent  and  projected  or  planned  ownship  activities. 

The  Situation  Assessment  subsystem  combines  a  need  to  react  to  unexpected  objects  and 
events  with  the  query-driven  function  of  interrogating  the  external  space  with  respect  to  current 
or  proposed  plans,  and  in  response  lo  pilot  requests.  This  requires  that  the  subsystem  not  only 
reason  about  the  situation  in  a  data-driven  way,  but  that  it  is  also  capable  of  focusing  its  attention 
on  the  data  that  are  relevant  to  the  current  tactical  plans.  The  Situation  Assessment  function  in¬ 
cludes  methods  for  reasoning  with  uncertainty,  allowing  preliminary  conclusions  to  be  drawn 
based  on  imperfect  or  suspect  uata,  and  then  be  confirmed  as  contributing  data  is  available  and 
inferences  become  more  certain. 
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Mission  Planner 

The  Mission  Planner  creates  and  maintains  a  takeoff-to-landing  mission  profile,  including 
routes,  resources,  and  time  constraints,  which  establishes  the  context  for  all  Pilot’s  Associate  rea¬ 
soning.  Beginning  with  a  start  and  end  point,  and  available  fuel  and  other  resources,  it  generates 
a  flight  profile,  including  route,  that  trades  mission  performance  and  exposure  to  known  threats 
with  consumption  of  resources.  The  resulting  space-time  description  of  the  route,  including  pro¬ 
jected  threat  exposure  and  expected  consumption  budgets,  is  then  posted  for  other  subsystems  to 
reason  about.  The  Mission  Planner  also  functions  as  a  higher  level  planning  resource  for  the 
Tactics  Planner,  providing  minimum  risk  trajectories  to  support  both  offensive  and  defensive  tac¬ 
tical  plans. 


Tactics  Planner 

The  Tactics  Planner  subsystem  reasons  about  threats  and  targets  for  attack  or  evasion,  and 
tasks  the  Situation  Assessment  subsystem  to  monitor  certain  high  interest  target  data  macks.  The 
Tactics  Planner  does  not  invent  tactics.  It  recommends  tactics  suitable  to  the  actual  situation,  but 
which  are  selected  from  the  pilot's  own  pre-bnefed  tactics  or  or  from  a  library  of  stored  tactics 
The  selected  tactic  is  then  customized  to  the  geometry  and  timing  of  the  existing  situation. 
Approved  tactics  are  monitored  and  adjusted,  or  repealed  and  replaced,  in  response  to  enemy 
counter-moves. 

The  Tactics  Planner  maintains  a  set  of  viable  response  plans  to  situations  that  unfold  as  the 
mission  progresses.  It  is  based  on  a  skeletal  planning  scheme  in  which  a  hierarchy  of  plan  ele¬ 
ments  is  maintained.  Each  plan  element  carries  its  own  knowledge  base  of  when  it  should  be  in¬ 
voked,  how  it  is  specialized,  what  other  plan  elements  are  related,  what  actions  that  are  necessary 
for  execution,  and  the  conditions  for  failure  or  completion.  This  data  structure  permits  the  sys¬ 
tem  to  support  both  sequential  and  simultaneous  plans,  and  allows  for  multiple  plans  to  be  con¬ 
sidered  and  maintained  as  ready  options  until  the  mission  situation  dictates  the  selection  of  an 
exclusive  option.  The  heirarchial  structure  permits  the  invocation  of  general  plans  even  before 
precise  details  of  the  situation  are  known.  Moreover,  the  distributed  knowledge  base  allows  a 
plan  to  be  modified  pieccwise-at  the  plan  element  level-withoul  destroying  the  larger  structure 
of  the  plan  The  Tactics  Planner  also  includes  a  set  of  reflexive  responses  to  urgent  situations  to 
support  rapid  response  to  previously  undetected  threats 

Lockheed’s  Program  Approach 

From  the  outset,  it  was  recognized  that,  the  engineering  methodology  used  to  develop  the 
Pilot’s  Associate  could  be  as  important  as  the  resulting  system.  No  similar  large-scale  real-time 
intelligent  system  development  had  previously  been  attempted,  and  the  degree  to  which  previous 
expenence  with  expert  systems,  coupled  with  conventional  avionics  software,  would  transfer  to 
the  Pilot’s  Associate  was  unknown.  The  program  presented  sene  additional  difficult  challenges. 
There  was,  for  example,  no  sufficiently  large  concentration  of  experienced  knowledge  engineers 
in  Lockheed  or  any  other  aerospace  company  to  accomplish  the  program.  Also,  the  operational 
domain  of  the  1995-era  fighter  aircraft  system  was  itself  largely  conjectural,  since  neither  the 
host  weapons  systent-an  advanced  fighter  aircraft-nor  its  employment  concept  existed  at  that 
time. 

A  number  of  aggressive  and  novel  development  techniques  were  adopted  to  cope  with  these 
challenges  and  were  flexibly  applied  as  the  program  evolved.  Over  time,  this  collection  of 
techniques  for  developing  intelligent  systems  has  evolved  into  a  systematic  process.  It  is 
generally  accepted  that  much  of  the  success  of  the  program,  certainly  with  resoect  to  the  team’s 
achievement  within  cost  and  schedule,  can  be  attributed  to  this  resulting  development  method.  It 
is  anticipated  that  the  process  will  reach  a  degree  of  maturity  in  the  current  phase  of  the  Pilot’s 
Associate  that  will  allow  its  generalization  and  application  to  the  full  life  cycle  development  of  a 
broad  class  of  intelligent  associate  systems. 

Structured  Rapid  Prototyping 

A  key  aspect  of  the  methodology  is  the  use  and  function  of  rapid  prototyping.  Early  design 
analyses  and  knowledge  engineering  efforts  indicated  that  the  conventional  "waterfall”  software 
development  method,  dependant  on  and  flowing  from  a  concise  set  of  well-defined  system 
requirements,  could  probably  be  modified  and  augmented  for  the  Pilot’s  Associate  program.  The 
absence  of  any  comparable  precedent  system  suggested  that  it  would  be  necessary  to  include 
frequent  evaluations  in  the  form  of  system-level  prototypes  to  assess  progress  and  to  keep  the 
program  on  track. 


The  very  circumstances  that  recommend  an  intelligent  avionics  system  seem  to  make  it 
impractical  to  analytically  define  a  precise  set  of  requirements  that  specify  the  system  in  a 
conventional  manner.  The  chosen  program  approach  was  to  describe  an  initial,  relatively  small, 
set  of  hypothetical  requirements  and  test  and  expand  them  with  a  series  of  prototype  systems. 
Results  flowing  from  each  prototype  would  then  form  the  basis  for  a  new  set  of  requirements  and 
subsequent  prototype.  This  general  process  had  been  successfully  used  in  the  development  of 
many  experimental  expert  systems.  To  make  it  suitable  for  a  larger  scale  engineering  program,  it 
was  necessary  to  structure  the  prototype  cycle,  adding  a  defined  set  of  top-level  objectives  and  a 
formal  sequence  of  development  activities,  as  depicted  in  Figure  3,  to  each  planned  three-month 
cycle. 


The  structured  rapid  prototype  cycle  has  proven  to  be  an  invaluable  feature  of  the 
development  process.  The  incremental  system  prototypes  have  become  much  more  than 
progress  checks  and  debugging  testbeds,  they  represent  the  principal  expression  of  the  actual 
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system  concept.  Thus,  experience  gained  in  the  implementation  of  each  prototype  provides  both 
guidance  and  motivation  for  refining  and  illustrating  requirements  and  design  for  succeeding 
prototypes.  Often,  they  have  also  illuminated  significant  new  requirements  or  subtle 
prerequisites  to  existing  ones.  They  serve  as  a  primary  vehicle  for  communicanon  within  the 
team  and  with  outside  observers.  The  prototypes  also  add  to  the  knowledge  engineering  process, 
serving  as  a  springboard  and  focus  for  knowledge  acquisition  activities.  Of  particular  interest  is 
the  insight  the  prototypes  provide  to  the  domain  expens  of  the  potential  employment  of  the 
system  and  how  such  a  capability  would  affect  their  actions. 
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Figure  3:  Typical  Rapid  Prototype  Cycle 


Advisory  Boards 

Two  Advisory  Boards  were  formed  to  provide  objecnve  advice  and  support  from  distinct 
perspectives.  The  Technical  Advisory  Board  is  comprised  of  seven  consultants  with  national 
reputations  in  various  aspects  of  Artificial  Intelligence  systems  development.  This  board 
reviews,  at  one  subsystem  per  quarter  year,  each  major  element  of  the  Pilot’s  Associate  system. 
By  this  process  of  “defending  the  program  thesis"  on  a  regular  basis,  the  program  managers  were 
able  to  minimize  technical  nsk,  exploit  opportunities  in  parallel  research  and  identify  potential 
problems  before  they  become  significant. 

The  Operational  Task  Force  is  comprised  of  former  operational  experts  (both  USAF  and  US 
Navy  fighter  pilots)  with  recent  experience,  now  in  industry,  who  review  the  operational  aspects 
of  the  program.  This  group  serves  as  a  liaison  with  the  operational  community  and  ensures  that 
their  expressed  concerns  are  appropriately  addressed.  It  also  serves  to  focus  knowledge  acquisi¬ 
tion  activities  and  acts  as  a  forum  for  resolving  differences  of  opinion  among  experts  and 
developing  a  coherent  body  of  expert  knowledge  for  inclusion  in  the  Pilot’s  Associate 

From  a  program  management  perspective,  the  advisory  boards  have  been  helpful  in  finding  a 
useful  balance  between  immediate  development  concerns  and  the  long  term  view  of  the  program 
leading  to  transition  of  the  technology  into  mission  aircraft. 

Distributed  Team  Concept 

Recognizing  that  there  was  not  sufficient  technical  ability  in  the  emerging  field  of  artificial 
intelligence  within  even  a  large  aerospace  corporabon  like  Lockheed,  a  relatively  large  team  of 
engineers  was  assembled  from  several  companies  located  throughout  the  United  States.  By  dis¬ 
tributing  technical  leadership  and  responsibility  for  subsystem  design  and  development  across  all 
the  participants,  Lockheed  has  fostered  a  cohesive,  goal-oriented  team.  In  severed  cases,  a 
company  leads  development  in  one  subsystem  and  supports  work  in  others.  This  arrangement 
promotes  both  enthusiastic  “ownership”  of  the  work  and  accountability  among  the  team 
members  and  stimulates  technology  sharing  and  cooperation.  In  general,  the  quarterly  prototype 
integration  sessions  have  provided  an  adequate  forum  for  personal  interactions  among  the 
engineers  and  government  program  participants.  Other  communicauons  have  been  managed 
largely  by  electronic  mail  (Arpanet  and  a  Lockheed  dedicated  computer  network)  and 
conventional  telephones. 


Evolution  of  “Intelligent  System”  and  “Associate  System”  Concepts 

An  important  early  product  of  the  of  the  Pilot’s  Associate  Program  was  a  more  refined  view 
of  the  nature  of  intelligent  avionics  systems  in  general,  and  of  aircrew  associate  systems  in  par¬ 
ticular.  This  maturing  view  has  greatly  influenced  both  our  understanding  of  the  potential  appli¬ 
cation  and  value  of  such  systems,  and  the  process  by  which  we  develop  them. 

Significant  Characteristics  of  Intelligent  Avionics  Systems 

Intelligent  avionics  systems  are  distinguished  from  expert  systems,  or  aggregations  of  expert 
systems  by  several  important  characteristics.  The  first,  and  perhaps  most  dramatic  difference,  is 
the  scope  of  the  application  domain.  Expert  systems  typically  focus  on  specific  problems  or  sets 
of  related  problems  within  a  single  well-bounded,  narrowly  defined  domain.  Intelligent  avionics 
systems,  conversely,  are  developed  to  cope  with  a  wide  range  of  dynamic  (in  space  and  time) 


problems,  general  in  nature  and  later  specified  by  actual  mission  context,  that  span  the  complete 
operational  domain  of  the  intended  user.  This  broader  application  necessarily  emphasizes  do¬ 
main-specific  problem  solving  techniques  and  strategic  knowledge.  It  forces  engineers  at  every 
level,  from  design  to  systems  implementation,  to  comprehend  and  capture  significant  aspects  of 
underlying  domain  theory  and  to  produce  a  flexible,  adaptable  product  capable  of  coping  with  a 
myriad  of  novel  situations  not  explicitly  considered  in  the  developmeni  process. 

A  second  significant  distinction  between  expert  systems  and  intelligent  avionics  systems  lies 
in  the  operating  environment  and,  in  particular,  in  the  manner  in  which  mission  data  are  acquired 
by  the  system.  Expert  systems  are  often  "consulting"  systems,  typically  designed  to  operate  in  a 
relatively  benign  environment.  Operating  knowledge  is  derived  through  a  dialogue  with  the  user 
that  depends  exclusively  on  his  ability  to  acquire  and  to  some  degree,  interpret  informauon  about 
the  problem.  Intelligent  avionics  systems  are  intended  to  operate  in  demanding  performance  en- 
vironments-fighter  aircraft  cockpits  in  combat  conditions,  for  example-that  may  not  present  re¬ 
liable  opportunities  for  such  a  dialogue,  and  that  may  render  such  a  procedure  infeasible. 
Therefore,  intelligent  avionics  systems  must  obtain  much  of  their  operating  data  directly  from  a 
sensor  suite,  that  may  include  the  full  set  of  external  and  internal  sensors  aboard  the  host  plat¬ 
form,  without  human  intervention  or  interpretauon.  The  Pilot's  Associate,  for  example,  incorpo¬ 
rates  knowledge  and  strategies  for  obtaining  required  information,  as  well  as  the  ability  to  con¬ 
trol  sensors  and  an  awareness  of  the  consequences  of  those  actions.  Intelligent  avionics  systems 
must  also  incorporate  the  means  for  clear,  unambiguous  and  reliable  communication  with  the 
human  operator  in  environments  that  obviate  more  tradiuonal  single  channel  input/output  devic¬ 
es.  This  important  feature  of  intelligent  avionics  systems  plays  a  large  role  in  reducing  operator 
workload  and  in  producing  a  speedy,  efficient  man-machine  interface 

Additionally,  the  demanding  operational  environment  implies  a  robust  system  that  functions 
dependably  with  incomplete,  noisy  or  even  false  data.  It  adds  requirements  for  a  system  that  can 
operate  on  rugged  host  processors  and  that  can  be  used  to  advantage  by  average  or  even  entry- 
level  operators.  Considerable  care  is  required  in  the  development  of  the  user  interface  and  in  the 
inclusion  of  internal  “sanity  checks”  to  accommodate  the  typical  errors  of  novice  users 

Finally,  intelligent  avionics  systems  are  distinguished  from  expert  systems  in  their  degree  of 
interaction  with  other  electronic  or  mechanical  systems  Conventional  expert  systems  are  often 
stand-alone  systems.  Intelligent  avionics  systems  are  conceived  as  an  element  of  a  larger,  inte¬ 
grated  electronic  system.  And  as  such,  a  primary  role  of  an  intelligent  system  is  to  provide  func¬ 
tional  integration  cf  various  electronic  systems  in  order  to  bring  the  fullest  capability  of  the  total 
system  to  bear  on  particular  problems,  in  the  context  of  actual  circumstances.  That  capability  is 
principally  derived  from  the  particular  ability  of  intelligent  avionics  systems  to  be  aware  of  the 
actual  situation  and  to  understand  and  reason  about  objectives 

Significant  Characteristics  of  Associate  Avionics  Systems 

In  a  similar  manner,  our  view  of  a'  sociate  avionics  systems,  as  a  particular  class  of 
intelligent  avionics  systems,  has  cm!  ed  considerabl; .  Initially,  an  associate  system  was  seen  as 
an  essentially  self-contained,  independent  system  that  performed  a  well-defined  set  of  operations 
in  response  to  specific  requests  to  support  a  particular  set  of  human  activiues. 

We  discovered  that  this,  seemingly  reasonable,  perspective  of  the  system  was  inconsistent  with 
some  important  realities  of  human  nature  and  the  operational  environment.  In  particular,  two  sa¬ 
lient  operational  requirements  forced  a  revision  of  our  view  of  the  Pilot’s  Associate  and,  conse¬ 
quently,  of  associate  systems  in  general. 
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Figure  4-  Impersonal,  Unsympathetic  Model  Associate  System 


The  first  environment,  shown  schematically  in  figure  4,  resulted  from  the  tactical  require¬ 
ment  to  devise  a  system  that  would  function  and  produce  results  in  a  fashion  that  would  be  pre¬ 
dictable  to  the  associate  human  (i.e.,  the  pilot),  but  unpredictable  to  his  adversary.  We  quickly 
realized  that  the  system  must  consistently  furnish  the  pilot  with  results  that  are  believed  to  be 
"reasonable”  (i.e.  that  are  predictable  to  the  pilot)  or  they  would  be  ignored.  At  the  same  time,  it 
is  critically  important  that  those  results  do  not  force  a  stereotyped  set  of  predictable  actions  that 
yield  substantial  advantages  to  opponents. 

The  second  operational  environment,  shown  in  figure  5,  flows  from  the  exigencies  of  the 
performance  requirement  which  demand  an  extremely  efficient  and  precise  human  interface,  one 


that  frequently  depends  on  implicit  communication.  Often  decisions  must  be  made  and  actions 
taken  in  time  periods  too  short  for  a  two-way  exchange  of  information.  In  other  situations,  phys¬ 
ical  or  tactical  constraints  impede  verbal  or  manual  communication.  In  these  cases,  critical  com¬ 
munications  are  embedded  in  overt  pilot  actions  or  his  lack  of  action.  Typically,  correct  interpre¬ 
tation  of  such  actions  by  an  associate  system  depends  on  the  understanding  of  an  established  plan 
for  these  circumstances. 

Consider,  for  example,  the  situation  of  a  fighter  pilot  in  an  hypothetical  future  conflict, 
equipped  with  weapons  effective  at  ranges  greater  than  thirty  nautical  miles,  and  a  multi-spectral 
sensor  suite  effective  at  perhaps  twice  those  ranges,  confronting  a  similarly  equipped  opponent. 
Given  the  large  detection  volume  of  such  sensors,  and  the  projected  density  of  air  traffic  in  future 
conflicts,  it  seems  likely  that  the  pilot  will  frequently  encounter  situations  in  which  the  number 
of  possible  threats  or  targets  within  or  nearing  critical  range  exceed  the  number  of  air-air  missiles 
on  board  (and  probably  the  combined  total  of  weapons  in  a  tactical  element).  The  pilot  will  then 
be  required,  in  a  very  short  time,  to  make  a  series  of  decisions  that  balance  mission  objectives 
and  risk  to  the  group  and  himself,  that  consider  complex  rules  of  engagement,  resources  and  fu¬ 
ture  engagements,  and  that  frequently  depend  on  subtle  tactical  clues  and  fine  detail.  When  the 
situation  is  further  complicated  by  even  simple  enemy  tactical  maneuvers  and  countermeasures, 
the  decision  process  quickly  reaches  a  level  that  may  require  decision-aiding  systems  for  the 
pilot.  The  crucial  quality  of  such  assistance  in  a  given  situation,  whether  provided  by  a  human 
crew  member  or  an  electronic  system,  is  the  correct  anticipation  of  the  pilot’s  needs  that  results 
in  the  timely  provision  of  the  “right"  information  or  initiation  of  some  particular  action  to  sup¬ 
port  the  pilot’s  planned  response  to  the  situation  In  these  cases,  “right"  is  highly  individualistic. 
Some  pilot’s  focus  on  some  specific  data,  others  rely  on  an  awareness  of  the  whole  situation; 
some  pilots  are  bold  and  aggressive;  others  cautious  and  cunning.  A  “standard’  set  of  informa¬ 
tion  or  actions  that  forces  the  pilot  to  adapt  his  concept  of  action  to  some  other  view  of  the  situa¬ 
tion  is  distracting  at  best  and  frequently  detrimental  Even  worse  is  a  process  that  mechanically 
results  in  the  same  response  to  every  situation  and  may  provide  a  substantial  tactical  advantage 
to  the  enemy 

A  highly  personalized  tactical  system,  that  incorporates  the  pilot’s  personal  plans  and  re¬ 
sponses  to  a  variety  of  anticipated  situations,  and  that  can  further  specialize  them  according  to 
the  pilot’s  own  rules  for  specialization,  offers  a  solution  to  the  dual  problems  of  providing  an  ef¬ 
ficient  interface  and  tactical  variation.  This  is  shown  in  figure  5.  The  current  implementation  of 
this  concept  for  the  Lockheed  team’s  Pilot’s  Associate  is  realized  by  means  of  a  ground-based 
“Mission  Support  Tool”  that  allows  the  pilot  to  customize  selected  parameters  of  the  airborne 
Pilot’s  Associate  to  his  personal  preferences,  and  to  incorporate  his  own  plan  for  the  current  mis¬ 
sion.  Thus,  the  pilot  maintains  the  responsibility  for  planning  the  mission  and  selecting  or  deriv¬ 
ing  the  pnmaiy,  secondary  and  contingency  tactics  for  a  number  of  expected  situauons.  The 
Pilot’s  Associate  is  configured  as  an  instrument  to  support  the  precise  execution  of  those  plans, 
when  they  are  appropriate,  or  to  select  a  tactic  from  the  pilot’s  own  stored  library  of  tactics  when 
the  pre-selected  briefed  tactics  are  unsuitable.  In  the  heat  of  combat,  the  pilot  will  not  b<-.  forced 
to  try  to  make  sense  of  some  novel  scheme  invented  by  a  machine;  instead  he  will  only  be  pre¬ 
sented  with  precise  specializations  of  plans,  appropriate  to  the  actual  situanon,  that  he  analyzed, 
selected  and  briefed  with  his  wingmen  for  the  mission. 


Figure  5:  Personal,  Aggressive  Support  System  Model 

The  feasibility  of  this  concept,  which  is  strongly  endorsed  by  the  team  domain  experts  and  our 
Operational  Task  Force,  was  illustrated  with  the  Tactics  Planner  subsystem  in  the  most  recent 
formal  system  demonstration  in  November  on  198®  In  that  case,  the  engagement  tactics  for  the 
demonstration  mission  were  developed  on  an  early  prototype  Mission  Support  Tool  and  subse¬ 
quently  loaded  into  the  Pilot’s  Associate  for  successful  use  in  the  demonstration. 

Summary 

The  Pilot’s  Associate  effort  pursued  by  the  Lockheed  development  team  has  produced  not 
only  prototypes  of  intelligent  avionics  systems,  but  also  new  insights  imo  the  nature  of  these 
systems,  and,  new  techniques  for  designing  and  developing  software  for  these  and  similar 
systems.  The  structured  rapid  prototyping  methods  that  have  been  utilized  have  also  provided 
the  ultimate  consumers-tactical  pilots-with  significant  awareness  of  the  design  concept  and 
operational  value  of  the  Pilot’s  Associate,  and  with  many  opportunities  to  improve  the 
requirements  for  such  a  system.  It  is  believed  that  this  fusion  of  operator  input,  through  the 
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knowledge  engineering  process,  and  intelligent  systems  technology  will  result  in  an  avionics 
system  that  will  provide  authentic  situation  awareness  of  all  factors  relevant  to  the  assigned 
mission  and  provide  dramatic  improvements  in  mission  effectiveness  and  aircraft  survivability. 
The  fielded  version  of  the  Pilot’s  Associate  will  eventually  allow  access  to,  and  flexible 
employment  of,  every  aircraft  system,  placing  a  floor,  not  a  ceiling,  on  the  pilot’s  performance, 
providing  him  die  crucial  edge  in  future  air  combat. 
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Cette  communication  fait  suite  aux  travaux  rialisis  dans  le  cadre  des  marches  DRET 

N"  85/400  et  87/433. 

Risumi :  Sur  tous  los  vihicules  militaires,  avions,  drdnes,  bateaux,  blmdis  et 

robots  terrestres,  I’imagerie  mfrarouge  s'impose  chaque  jour  de  plus  en 
plus  en  raison  de  ('information  supplimentaire  qu'elle  apporie  de  jour 
comme  de  nuit,  par  tout  temps  et  en  toute  discretion. 

Un  systdme  d’aide  i  I'interpritation  de  ce  type  d’images  a  it§  itudie  II 
utilise  notamment  des  connaissances  de  navigation,  de  perception  en 
mfrarouge  et  de  mise  en  oeuvre  de  traitement  d’image  pour  faciliter 
I'interpritation  en  permettant  la  prise  en  compte  optimale  des  donnies 
prisentes  i  bord  :  carte  numirique  du  terrain,  syst6me  de  navigation 
inertielle,  informations  mitiorologiques.  Ces  travaux  ont  iti  concretises 
par  la  realisation  d'une  maquette  en  laboratoire. 

Mots  cles  :  Systdme  a  Base  de  Connaissances,  Interpretation  d'lmages, 

Navigation  Inertielle,  Cartographic  Numirique,  Infrarouge. 

Abstract :  Infrared  sensors  are  becoming  more  and  more  essential  on  all  military 
vehicles  :  aircrafts,  RPVs,  boats,  tanks  and  land  robots,  because  of  the 
extra  information  they  provide,  day  and  night,  in  all  weather  conditions 
and  with  absolute  discretion. 

This  paper  presents  a  system  designed  to  help  in  the  interpretation  of 
infrared  images.  It  uses  navigation,  infrared  sensing,  and  image 
processing  knowledge  to  make  interpretation  easier,  allowing  optimal  use 
of  onboard  data  :  digital  maps,  inertial  navigation  system,  weather 
information.  This  work  has  been  validated  on  a  lab  prototype 

Keywords  :  Knowledge  Based  System,  Image  Interpretation,  Inertial  Navigation, 
Digital  Maps,  Infrared. 
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1.  INTRODUCTION 


I'dventail  des  systdmes  optroniques  en  production  ou  en  cours  de  d6veloppement 
couvre  les  trois  grands  domaines  :  terra,  air,  mer.  D’un  point  de  vue  op6rationnel,  sur 
appareil  pilots,  deux  besoins  au  moins  apparaissent  avec  I'utilisation  d’lmages 
infrarouges  : 

-  Aide  4  la  lecture  des  images  par  la  fourniture  au  pilots  d'une  image  avec  incrustation, 
par  imagene  de  synthase,  des  rdsultats  de  ('interpretation, 

-  Aide  ou  automatisation  du  recalage  par  designation  d'amer. 

L'etude  presentee  ici,  qui  a  6te  concretises  par  la  realisation  d'une  maquette 
laboratoire,  a  demontre  la  faisabilite  et  I'interet  operationnel  d'un  systems  e  base  de 
connaissances  d'aide  e  l'interpr6tation  d’images  infrarouges  a  bord  d'un  v6hicule.  Cette 
interpretation  d'images  infrarouges  (8pm  -12pm),  se  fait  sur  la  base  de  trois  types 
d'mformations : 

-  localisation  et  attitudes  du  vehicule  fournies  par  un  systems  inertiel  de  navigation, 

-  informations  geographiques  (planimetrie  et  attimetne), 

-  informations  sur  les  parametres  r6gissant  le  comportement  thermique  des  elements 
des  scenes  e  interpreter. 

En  pratique,  4  bord  d'un  vehicule  militaire,  ('interpretation  d'images  comports  deux 
fonctions  qui  correspondent  4  deux  probiemes  assez  differents  d'un  point  de  vue 
cogmtif : 

-  Une  fonction  de  Navigation,  ou  I'on  cherche  4  retrouver  dans  le  paysage  les 
elements  les  plus  caract6nstiques  portes  sur  les  cartes  et  que  I’on  pense  visibles 
compte  tenu  de  la  position  estimee.  Cette  demarche  se  caract6rise  par  un  processus 
de  raisonnement  descendant  et  un  domains  de  connaissances  bien  deiimite  (univers 
restreint  dans  lequel  les  symboles  4  identifier  sont  en  nombre  limite  et  connu) 

•  Une  fonction  d'Observatfon,  ou  I'on  analyse  1'image  afin  d'y  detecter  des  menaces 
pouvant  etre  des  elements  annonces  mais  non  localises  (colonne  de  blindes  en 
deplacement..),  voire  des  elements  totalement  impr6vus  (battens  de  DCA,...).  Cette 
approche  conduit  4  un  processus  de  raisonnement  ascendant  ou,  autrement  dit,  une 
demarche  guides  par  les  indices  trouves  dans  I’image.  L'univers  des  connaissances 
4  employer  est  ici  largement  ouvert,  du  fait  de  la  m6connaissance  du  type  et  du 
nombre  de  symboles  4  rechercher 

Dans  cette  etude,  I'attention  a  6t6  focalis6e  sur  le  mode  Navigation  et  un  cas  d'etude  a 
6t6  utilise,  celui  d'un  avion  d'armes  volant  4  basse  altitude. 


2.  STRUCTURE  DU  SYSTEME 


Une  caractenstique  fondamentale  du  problems  etudie  est  I'aspect  tr4s  descendant  des 
processus,  c'est-4-dire  du  raisonnement  de  la  carte  vers  I’image  soit  encore  du  modble 
vers  les  donn6es.  Le  terrain  survoie  6tant  en  effet  connu,  le  problems  est  de  se  situer 
dans  cet  univers. 

Afin  de  presenter  ('architecture  du  systems,  prenons  un  example  : 

0  Aun  Instant  donni,  on  salt  que  I'on  se  trouve  d  un  endrolt  donn6  et  que  I'on 
regarde  dans  une  certalne  direction ;  que  peut-on  voir  ? 

Un  module  d'Extraction  de  Carte  Active  a  pour  role  de  dresser  la  liste  des 
elements  portes  sur  la  carte  et  visibles  a  priori  compte  tenu  de  la  position  et  des 
attitudes  connues  inertiellement.  Pour  ces  elements  visibles  du  paysage,  que 
nous  nommerons  ici  Symboles,  il  determine  une  apparence  geometrique  a  pnori 
et  une  fenetre  de  recherche  dans  I'image. 


% 

% 
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0  On  volt  un  chdteau  d'eau  at  une  route,  lequel  des  deux  rechercher  en 
premier  ? 

Un  module  Navigation  va  decider  de  I'ordre  dans  lequel  la  recherche  des 
symbotes  sera  effectude.  II  applique  pour  cela  des  connaissances  de  Navigation 

Ce  module  va  dgalement  devoir  gdrer  la  reconnaissance  des  diffdrents  symboles 
d'une  m6me  scdne  ou  d'un  m§me  symbols  &  plusieurs  instants  successifs. 

0  Peut-on  en  savolrplus  sur  le  chateau  d'eau  ? 

Un  module  de  Derivation  va  prendre  en  compte  les  conditions  de  vol  et  tout 
particuli£rement  mdteorologiques  pour  completer  le  models  du  symbole  en 
introduisant  une  description  radiometrique  de  celui-ci  Les  connaissances 
appliquees  portent  done  sur  la  perception  en  infrarouge  &  basse  altitude. 

0  Comment  rechercher  ce  chateau  d'eau  ? 

Lors  de  la  premiere  phase  de  l'6tude,  nous  avions  panse  ddenre  les  symboles  en 
termes  d’indices  [CRE  88].  Les  essais  mends  sur  la  premiere  maquette  ont  montrd 
que  ce  type  de  decomposition  6tait  un  peu  trop  ngide  et  ne  correspondait  pas  a 
I'expertise  de  traitement  d'image.  Celle-ci  consiste  plutot  a  se  ramener  a  une 
descnption  plus  abstraite  de  I'objet,  sur  les  2  aspects  geometne  et  radiomdtne  et 
choisir  dynamiquement  les  traitements  les  plus  adaptds  e  partir  de  la  donnee  de 
cette  seule  description  abstraite.  II  est  done  apparu  une  notion  de  lili&res  de 
traitements  d'image  (contours,  regions,  morpho-math...)  permettant  de  r6soudre 
sp6cifiquement  certaines  classes  de  probiemes. 

La  recherche  d'un  symbole  est  done  realises  par  I'application  de  plusieurs  filipres 
successives  sous  le  contrdle  dynamique  d'un  module  Identification.  Les 
connaissances  appliquees  a  ce  niveau  sont  stnetement  du  domaine  de  la  vision 

0  Comment  mettre  en  oeuvre  la  fllldre  contours  cholsie  par  le  module 
Identification  ? 

Un  module  de  Contrfile  Vision  est  charge  de  decomposer  la  filidre  relenue  en 
op6rateurs  ei6mentaires,  de  les  param6trer  s’ll  y  a  lieu,  de  les  fairs  s'exdcuter,  de 
contrfiler  le  rdsuftat  de  leur  application  par  des  rnesures.  Les  connaissances 
appliquees  a  ce  niveau  sont  6gatement  du  domaine  de  la  vision. 

0  Enfln,  qul  va  falre  les  operations  sur  I'lmage  ? 

Un  module  d'Operateurs  Vision  va  ex6cuter  ces  operations.  II  est  base  sur  une 
biblioth6que  g6n6rale  de  traitements  d'image. 

0  Derntdre  question  :  comment  prendre  en  compte  les  rdsultats  de  traitement 
d'image  ? 

II  faut  un  mdcanisme  de  remontde  des  informations  jusqu'au  niveau  du  module 
Navigation.  Les  opdrateurs  de  vision  retournent  au  controls  vision  des  elements 
permettant  a  celui-ci  d'apprdcier  I'efficacite  de  leur  application  et  d'avoir  un 
controls  veritablement  adaptatif.  A  la  fin,  le  dernier  opdrateur  remonte  un  ou 
plusieurs  indices  que  le  Controle  Vision  remonte  directement  a  I'ldentification.  Le 
module  identification,  a  son  tour,  aprds  avoir  active  une  ou  plusieurs  filidres, 
remonte  au  module  Navigation  les  localisations  dans  I'image  correspondant  a  ces 
symboles. 

On  remarquera  sur  la  figure  2.1  les  trois  dtages  du  traitement . 

.  Traitements  g6om6triques 

.  Contrfile  intelligent  de  ('Interpretation  :  Les  modules  Navigation,  Derivation,  et 
Controle  Vision  ont  comme  point  commun  la  programmation  declarative 
.  Traitements  d'image 


or 
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Trait  •fn*nt« 
G*om4tHqu*« 


ContrM* 

tnUWgwtt 


Trait*  m«nt 
dtm*g* 


FIGURE  2.1  ARCHITECTURE  FONCTIONNELLE 
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3.  MODELISATION  DC  L'UNIVERS  D'INTERPRETATION 


3.1.  INTRODUCTION 

L'univers  d'interprdtation  est  I'ensemble  des  informations  necessaires  au 
fonctionnement  du  systeme. 

II  s'agit  d'une  part  des  donnSes  d'entr6e  du  systems  : 

-  les  informations  cartographiques, 

•  les  conditions  thermiques, 

-  les  informations  inerlielles, 

-  les  images. 

et  d'autre  part  des  r6sultats  de  I'interprdtation  (hypotheses  de  reconnaissances  de 
symbotes). 

Nous  ne  ddtaillerons  ici  que  les  informations  cartographiques  et  la  representation 
des  resultats  de  l'interpr6tation 


3.2.  LES  INFORMATIONS  CARTOGRAPHIQUES 

La  consultation  de  pilotes  d'une  part  et  de  sp6cialistes  en  traitement  d'lmages 
d'autre  part  nous  a  permis  de  d6finir  une  liste  de  symboles  dont  la  recherche 
presente  un  int6rel  sigmficatif.  Nous  nous  sommes  ainsi  limite  aux  seuls  elements  e 
la  fois  utiles  pour  la  navigation  et  potentiellement  d6tectables  par  des  techniques  de 
traitement  d’image. 

Pour  mod6liser  et  mampuler  les  symboles,  une  approche  objet  a  et6  utilisee,  car  ells 
offre  un  moyen  d'impl6mentation  direct  de  concepts,  tout  on  constituant  un 
environnement  de  programmation  efficace.  Cette  representation  porte  en  elle  une 
part  importante  de  la  connaissance  Ii6e  au  prcbieme,  de  par  la  structuration  des 
donn6es  et  les  m6camsmes  d'hentage  qu'elle  permet  de  r6aliser. 

Les  symboles  ont  done  6te  d6crits  sous  formb  d'une  taxinomie  qui  repose  sur  le 
concept  de  Symbole  : 


Symbole 


Volume  Etendue  Trongon  Croisement 


Trongon  droit  Trongon  courbe 


Les  travaux  men6s  en  vision  ont  fait  ressortir  que  I'export  en  traitement  d'image 
faisait  plutet  abstraction  de  I'objet  e  rechercher  et  s6lectionnait  ses  operateurs  en 
fonction  d'un  module  plus  abstrait  d6cnvant  simplement  le  symbole  sous  ses  aspects 
dimensionnels  et  radiometriques.  La  representation  du  symbole  comporte  done  des 
champs  d6crivant  ses  caracteristiques  dimensionnelles  et  radiometriques 
apparentes  dans  I'image  et  constituant  une  description  abstralte  du  symbole 

La  structure  qui  permet  de  representer  les  informations  geographique  est  la  carte 
symbolique,  ou  S-Carte,  qui  inclut  la  p!amm6trie  (liste  de  symboles)  et  I'altimetrie 
(modeie  numenque  de  terrain  GMEDTN) 


3.3.-LES  RESULTATS  DE  L'INTERPRETATION 


Pour  rdaliser  la  premiere  reconnaissance  d’un  symbole,  on  peut  utiliser  la  position 
donnde  par  I'inertie  pour  determiner  les  elements  g6omdtriques  permettant  la 
recherche  du  symbole  (position  dans  nmage,  forme,...). 

Cette  premiere  reconnaissance  va  nous  fournir  une  ou  ptusieurs  localisations 
possibles  dans  1'image  correspondant  e  autant  d'hypotheses  de  reconnaissances 
du  symbole. 

Comment  exploiter  ces  r6sultats  pour  faciliter  la  recherche  d'autres  symboles  et 
discrimmer  les  diff6rentes  remontees  ? 

1)  Sur  une  sequence  en  adoptant  des  modeiisations  de  deplacement  dans  I'image 
utilisdes  en  ooursuite  de  able  pour  verifier  si  tout  au  long  de  la  sequence  on 
arrive  a  poursuivre  chaque  echo  initial.  En  pnncipe,  si  Ton  se  rapproche  du 
symbole,  il  va  grossir  et  sa  discnmination  par  rapport  aux  artefacts  deviendra 
possible  et  seule  une  hypothese  devrait  subsister. 


2)  On  peut  alors  utiliser  un  raisonnemant  aeometnaue  oualitatif  •  "Compte  tenu  de 
I'axe  d’approche,  le  chateau  d'eau  de  Castelviel  est  obligatoirement  vu  a  gauche 
de  la  N14  ;  done,  comma  on  pense  avoir  reconnu  la  N14,  parmi  les  deux 
localisations  trouvdes  pour  le  chateau  d'eau  seul  I'echo  1  est  une  solution 
possible" 


Carte 

Image 

U 

■Vv'RC 

S  / 

Axe  d'approche/ 

[I  Localisatior^^^^ 

/  1 _ 

U  acceptable 

Aucune  de  ces  deux  solutions  n'est  pleinement  satisfaisante,  soil  parce  qu'elle  ne 
permet  pas  d'utiliser  le  rdsultat  d'une  reconnaissance  pour  faciliter  et  diagnostiquer 
la  reconnaissance  d'autres  symboles,  soit  parce  qu'elle  n'autorise  pas  la  pnse  en 
compte  de  I'aspect  "s6quence  d'lmages".  Nous  avons  done  pr6f6r6  une  solution  qui 
tire  profit  de  la  disponibilita  d'une  centrale  inertielle  a  bord  uu  vrihicule. 

Le  princIpe  retenu  dans  I'apptlcatlon  consiste  a  conslddrer  chaque 
Interpretation  comma  une  observation  de  I'erreur  de  la  navigation  Inertielle  de 
I'appareil.  A  chaque  interpretation  correspond  done  une  hypothdse  de  locd'isation 
rdelle  de  I'appareil  -  obtenue  en  corrigeant  la  position  inertielle  a  panir  de 
I'observation  rdalisde.  Lorsqu'une  interpretation  gdndre  plusieurs  possibility  de 
localisation  d’un  amer  dans  I'image,  cela  correspond  a  plusieurs  hypotheses  de 
localisation  de  I'appareil, 

Ce  principe  a  6td  impidmentd  grace  a  un  filtre  de  Kalman  dont  le  vecteur  d'dtat  a 
deux  composantes  :  3L,  I'erreur  inertielle  de  latitude  et  3G,  I'erreur  inertielle  de 
longitude. 

La  gestion  des  hypotheses  de  localisation  permet  la  prise  en  compte  de  remontees 
vision  donnant  plusieurs  localisations  possibles  pour  un  symbole  recherchd. 

Reprenons  maintenant  I'exemple  fourni  pour  le  raisonnement  geomdtrique.  Dans  le 
champ  de  vision,  nous  avons  un  trongon  de  route  nationals,  un  chateau  d'eau  et  un 
batiment  non  portd  sur  la  carte. 

Supposons  que  I'on  se  recale  tout  d'abord  sur  la  route,  I'incertitude  sur  la  position  se 
rdduit  de  (agon  ellipsoi'dale  autour  de  I'erreur  estimde  avec  le  grand  axe  dans  la 
direction  du  trongon. 

Si  I'on  cherche  ensuite  a  ddtecter  le  chateau  d'eau,  en  utilisant  I’incertitude  rddoite, 
seule  la  localisation  correspondant  vdritabtement  au  chateau  d'eau  sera  acceptde, 
I'artefact  provenant  du  batiment  sera  rejetd,  n'dtant  pas  compatible  avec  I’estimee 
d'erreur  correspondant  k  la  reconnaissance  de  la  route. 


Carte 


Ugne  de  vis6e  sur  le 

chateau  tfeau  Positions 


Evolution  de  I'estlmbe  d'erreur  inertlelle 


Incertitude 

,  Observation 

avant  recalag^^ 

\  j  autrongon 

/Erreur  rbelle 

f  \ 
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1 

\y 

wSmm 

Incertitude  j 

corrospondant  7 

4  (observation  ' 

VZ-*'  \  Incertitude 

|  'correspondant 

4 1’observatlon 

Ou  Irongon  <ju  chateau  d'eau 


Cette  representation  des  rbsultats  da  l'interpr6tation  permet  de  propager  le  b6n6fice 
de  la  reconnaissance  d'un  element,  d  la  fois  dans  la  scene  courante  et  dans  les 
scenes  suivantes,  car  le  rbsultat  de  ('interpretation  est  exploiter  par  rapport  d  la 
reference  inertlelle  et  non  directement  dans  1’image. 


4.  DESCRIPTION  DETAILLEE  DU  SYSTEME 


11,  MODULE  D’EXTRACTIQN.CE  CARTE  ACTIVE 

Le  module  d'Extraction  de  Carte  Active  a  pcur  role  de  determiner  quels  sont  les 
symboles  visibles,  quelle  est  leur  apparonce  gbometrique  dans  I'image  (position, 
forme,  dimensions,  orientation)  et  dans  quelle  fenbtre  de  I'image  ils  pourront  §tre 
rechercl-.es.  II  regoit  pour  cela  du  module  Navigation  les  informations  de  situation  de 
i'apparell  sous  forme  d'une  localisation  integrant  : 

-  la  position  (L,  G,  Z)  et  les  attitudes  (Cap,  Roulis,  Tangage),  fourmes  par  la 
navigation  inertielle, 

-  une  estimation  de  i'erreur  inertielle  courante,  fournie  soit  par  la  navigation 
optimaie  (hypothese  de  depart  du  systeme),  soit  par  le  systdme  d'interprbtat.'on  lui- 
meme,  d  partir  de  reconnaissances  de  symboles  d6jd  eftectubes. 

II  renvoie  d  ce  mbme  module  Navigation  la  liste  des  symboles  visibles,  avec  leurs 
modeles  abstraits  renseignbs  du  point  de  vue  des  caractbristiques  gbombtriques 
apparentes  dans  I’image,  Cela  permet  de  constituer  une  image  predictive  de  la 
geometric  des  symboles  dans  I'image. 


Le  role  du  module  Navigation  est  la  geslion  globale  o'?  ('interpretation,  d  savoir 
decider  quels  sont  les  symboles  intbressants  d  reconnaitre  et  exploiter  les  rbsultats 
de  'eur  recherche  dans  I'image 

En  entrbe,  le  module  Navigation  regoit  ia  carte  active,  i.e.  la  liste  des  sy  -iboles 
supposes  visibles  compte  te.nu  da  ia  localisation  optimaie  de  I'appareil  li  regoit 
bgalement  I'ensemble  des  hypotheses  de  localisation  courantes. 

En  sortie,  aprbs  avoir  recherche  tous  les  symboles  utiles,  le  module  retourne  une  ou 
plusieurs  hypotheses  oe  localisation  (de  I'appareil),  en  principe  affinbes  par  rapport 
d  cellos  regues  en  entrbe. 
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La  module  Navigation  assure  ce  rfile  au  travers  de  deux  fonctions  prlncipales  : 
Elaboration  d'uno  Interpretation  complete 

Le  principe  d'eiaboration  d'une  reconnaissance  bar. 6  sur  le  concept  de  Localisation 
est  le  suivant  (voir  figure  4.1) : 

1)  On  prend  le  symbols  le  plus  interessant  de  la  premiere  scene  et  on  le  recherche 
en  fonction  de  la  localisation  inertielle  optimala.  On  obtient  une  ou  plusieurs 
hypotheses  de  localisations. 

2)  On  continue  ainsi  pour  tous  les  symooles  de  la  premiere  scene  puis  ceux  de  la 
scene  suivante  (dans  le  cas  ou  I'on  scuhaite  analyser  une  sequence)  en  prenant 
en  compte  e  chaque  fois  toutes  les  hypotheses  de  localisation.  II  se  constitue 
ainsi  un  graphs  de  localisations. 

Pour  chaque  nouvelle  localisation  dans  le  graphe  de  localisation,  on  calcule  un 
factaur  de  certitude  global  selon  une  fo-.nule  derives  de  la  theone  des  croyances 
(Dempster  Shafer) 

L'interpr6tation  est  termin6e  lorsque  le  graphe  ne  comporte  plus  qu'une  branche, 
avoc  un  taux  de  conf'ance  et  une  precision  de  localisation  estim6e  jug6s  suffisants. 


Scene  N° 1 


Scene  N° 1 


Localisation  inertleileoptimale 


Symbols  N°1 

LdCT 

Lo22 

Symbols  N°2 

A"" 

Lo«21 

/\ 

A 

Symbols  N°1 

Loci  11  Loci  12 

Loc21 1  Loc212 

x  Echec  da  la  branche 

Figure  4.1  GRAPHE  DE  LOCALISATION 


P 


Cholx  des  symboles 

Les  connaissances  de  choix  des  amers  int6ressants  pour  les  recalages  e  vue  sont 
d6tenues  par  les  pilotes  et  navigateurs.  Notre  syst6me,  bien  que  base  sur  une 
recherche  des  symboles  par  traitement  d'image,  utilise  sensiblement  les  m§mes 
connaissances.  Celles-ci  peuvent  se  traduire  sous  la  forme  d’un  ensemble  de 
criteres  d6correies  deux  e  deux  (surface  apparente,  longueur,  confondabilite,.,.). 

Le  choix  des  symboles  est  realise  en  deux  Stapes  • 

1)  Selection  des  symboles  globa'ement  acceptables,  c'est-4-dire  satisfaisant  un 
minimum  chaque  critdre  -  impiement6e  par  un  mdcanisme  procedural  de  filtrages 
successifs  sur  chacun  des  criteres. 

2)  Tn  de  ces  symboles  retenus.  Face  £  ce  type  de  problems  les  algorithmes  de  tn 
sont  impuissants,  sauf  4  ^laborer  un  enters  global  base  sur  une  somme  pond6r6e 
de  tous  les  criteres  ei6mentaires.  Le  problems  d’un  tel  "amalgame”  est  de  ne  pas 
permettre  une  r6etle  prise  en  compte  de  chaque  enters.  Par  example  un  symbols 
r4pondant  bien  8  tous  les  enteres  sauf  un  pourra  passer  devant  un  autre  symbols 
simplement  bon  mais  sur  tous  les  enteres,  ce  qui  n'est  pas  forc6ment  souhaitable. 
Le  choix  s'est  done  porte  sur  des  techniques  d'aide  8  la  decision  [ROY  87]  qui 
se  sont  r6v4l6es  tout  4  fait  adaptees  et  correspondant  bien  e  I'osprit  "Intelligence 
Artificielle"  de  programmation  declarative  et  de  manipulation  de  I’imprecis 


AA.  MODULE  DE  DERIVATION  DU  SYMBOLE 

Le  module  Derivation  a  pour  rfile  d'etablir  un  models  d6nv6  du  symbols  en  fonction 
des  conditions  de  vol  rencontrees.  II  s'applique  tout  particulierement  8  la  description 
radlometrique  du  symbols,  la  description  geometrique  ayant  d6j8  etablie  en  grande 
partie  par  le  module  d'Extraction  de  Carle  Active.  II  permet,  en  fonction  des 
conditions  rencontrees,  d'mstancier  ou  de  modifier  les  champs  de  description 
thermiques  du  symbols. 
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Ce  complement  d'mformation  est  ndcessaire  pour  afliner  le  cltoix  des  filidres  de 
vision  amsi  quo  le  paramdtrage  des  opdrateurs  qui  seront  appliques  par  la  suite 

C’est  un  systems  de  production  dont  la  base  de  regies  contient  les  connaissances 
relatives  d  I'mtluence  des  condition ?  de  vol  sur  I'aspect  du  symbole  et  dont  la  base 
de  (aits  est  constitude,  entre  act  ns,  par  le  symbole  d  ddriver  et  les  conditions 
mdtdorologiques  passees  et  pr6sei.<es. 

L'expertise  a  ete  fournie  par  des  spdcialistes  des  cameras  thermiques  et  par  analyse 
statistique  d'une  base  donndes  d'images  constitude  dans  le  cadre  du  projet. 


4.5.  LE  MODULE  OPERATEURS  VISION 

Le  role  du  module  Opdrateurs  Vision  est  de  mettre  a  la  disposition  du  module 
Contrdle  Vision  un  ensemble  d'operateurs  de  traitement  d’images  permettant  la 
detection  des  symboles  dans  les  images.  Ces  opdrateurs  sont  de  3  types  : 

-  opdrateurs  de  pr6-traitement,  transformant  une  image  en  une  autre  image, 

•  opdrateurs  de  traitement,  transformant  une  image  en  un  ensemble  d'mdices 
visuels, 

-  opdrateurs  de  post-traitement,  transformant  un  ensemble  d’mdices  visuels  9n  un 
ensemble  rdduit  de  ces  rr.'imes  indices  ou  en  un  ensemble  d’autres  indices 
visuels  de  plus  ht,ut  niveau 

Nous  ne  ddtaillerons  pas  ce  module  plus  avant,  nous  rappellerons  simpiement  deux 
concepts  trds  importants  de  is  le  systdme  : 

.  Le  moddle  abstralt  du  symbole  (gdomdtrique  et  radiomdtrique) 

C’est  d  lui  que  se  ramdnent  les  traitements  de  fagon  &  permettre  une 
'reprdsentation  standard'  des  rdsultats,  autorisant  ainsi  leur  dvaluation,  tri  et 
comparaison. 

.  La  reprdsentation  de  I'lmprdcls 

Pour  traduire  I’.mprdcision  lide  au  moddle  ddrivd  sur  la  base  duquel  sont  rdalisdes 
toutes  les  recherches,  une  structure  d’tntervalle  [  Mini,  Probable,  Maxi  ]  a  dtd 
utilisde.  Malgrd  sa  simplicity  ce  principe  s'est  rdvdld  suffisant. 

L’ensemble  des  opdrateurs  ddveloppds  est  prdsentd  dans  la  figure  4.2  sous  forme 
de  graphs  des  em-  atnements  possibles. 


K^AUQN 


Le  module  idem  tication  est  en  charge  de  la  plamfication  dynamique  et  du  controle 
de  la  recherche  te  symbole  dans  I'image.  L’identification  d'un  symbole  donnd  se  fait 
par  agrdgation  de  rdsultats  issus  de  recherches  par  traitement  d’lmage  conduites 
intdgralement  par  le  module  "Controle  Vision".  Chaque  recherche  correspond  au 
suivi  d'un  chemm  dans  le  graphs  d'opdrateurs  avec  a  priori  un  certain  nombre 
d'alternatives  locales  (voir  figure  4.3). 

Chaque  sous  graphs  correspondant  d  un  ensemble  cohdrent  d'opdrateurs 
permettant  de  mettre  en  dvidence  une  certaine  apparence  de  symbole  est  appeld 
Filldre.  Dans  le  cadre  du  projet,  sept  filidres  ont  dtd  ddveloppdes. 

Le  module  assure  trois  fonctions  que  nous  prdsentons  successivement  ci-aprds. 

Cholx  de  filldre 

II  est  assurd  par  un  systdme  de  production  dont  les  rdgles  component  en  padie 
gauche  des  conditions  sur  I’apparence  du  symbole  et  en  partie  droite  des  predicats 
mdiquant  I’intdrdt  des  diffdrentes  filidres  dans  ce  contexte.  Chaque  prddicat  se 
traduit  par  un  vote  recueiili  au  niveau  de  chaque  filidre  dans  une  structure  d  cinq 
champs  (de  non  satisfaisante  d  trds  satisfaisante). 

Quand  tous  les  votes  ont  dtd  rdalisds,  on  analyse  les  rdsultats  pour  dlimmer  les 
filidres  non  satisfaisantes  et  classer  par  ordre  d'intdret  "a  priori”  celles  qui  sont 
retenues. 


Activation  at  succession  das  fiildres 

La  communication  avec  la  niveau  infdrieur  "Contrdie  Vision"  se  tait  d  chaqua 
decision  d’application  d'une  filters,  sous  forme  d'activation  du  module  contrdie 
vision  pour  son  application  ;  en  retour  le  module  regoit  aucune,  une,  ou  plusieurs 
positions  possibles,  avec  pour  chacune  un  taux  de  confiance  "Vision".  Le  module 
lance  ensuite  la  filtere  suivante  ou  arrete  la  recherche. 

Evaluation  finale  das  remontdes 

L'application  de  plusieurs  filidres  successives  et  teurs  rdsultats  sont  pris  en  compte 
dans  le  calcul  d'un  facteur  de  confiance  retournd  au  module  Navigation.  Ce  calcul 
s'appuie  sur  la  comparaison  des  rdsultats  des  filteres  par  rapport  d  I'hypothese  de 
vision  de  depart,  mais  aussi  sur  la  comparaison  das  rdsultats  entre  les  difterentes 
filidres. 


4.7.  MODULE  CONTRQLE  VISION 

Le  module  Contrdie  Vision  est  activd  par  le  module  supdrieur  Identification  d'un 
Symbols,  dans  le  but  de  confirmer  ou  d'infirmer  la  presence  d'un  symbols,  sous  la 
forme  d'une  demands  duplication  d'une  filtere,  correspondant  d  la  recherche  d'un 
indice  visuel  donnd  et  tenant  compte  de  la  connaissance  a  prion  du  symbols 
contenue  dans  les  champs  gdomdtriques  et  radiomdtriques  constituant  son  moddle 
abstrait. 


Rdgions  Rdgions  p  .  An.  Segments//  Segments 
Bmaires  Morpho  Ngris  J _  Contours  Contours 


Image  Image  Binaire 
Binaire  depuis  Morpho  Ngris 


Passage  Reprds 
Standard  Rdgions 


/  \ 


Tri  final 


Image  de 
Pts  de  contraste 

uivi  +  Anti-Paralldles 
Extraction  de  I 

Segments  | 

Transformde 
de  Hough 


Projection  2 


Doublets  de 
Segments  // 


Passage  Reprds  Passage  Reprds  Passage  Reprds 
Standard  Standard  Doublets  Standard  Segments 


Tri  final 


Tn  final 


Tri  final 


Tri  final 


Figure  4.2  GRAPHE  DES  OPERATEURS  DE  VISION 
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Figure  4.3  EXEMPLE  DE  FIL1ERE 


La  conduits  d'une  filibre  Vision  consists  done  en  la  generation  d'une  sequence 
d'operateurs  de  traitement  d'image. 

Le  Controls  Vision  appalls  Is  niveau  interieur  de  Traitement  d'image  de  bas  niveau 
par  des  activations  successives  d'operateurs  vision  avec  la  donn6e  de  tous  les 
parambtres  necessaires  b  ces  op^rateurs. 

1  e  module  contrfile  vision  rbalisera  done  piusieurs  tbches  : 

-  determination  des  chatnes  d'operateurs  possibles  et  choix  de  la  plus  pertinents, 

-  guidage  de  I'application  des  opbrateurs,  notamment  le  choix  de  leurs  parambtros, 

-  integration  des  mesures  intermediaires  realisees  par  les  op6rateurs, 

-  test  des  conditions  d'6chec  de  la  recherche  apr6s  chaque  opbrateur. 


Le  concept  de  Macro-Op6rateur 

Le  Macro-Opbrateur  (note  MOP)  est  la  structure  de  base  du  module  Controle 
Vision.  Fonctionnellement,  un  MOP  traduit  une  entitb  en  termes  de  traitement 
d'image  [THO  88).  Ainsi,  on  considers  diffbrents  niveaux  de  MOP  allant  de 
l'op6rateur  vision  bas  niveau  e  ia  chains  de  traitement  complete. 

La  notion  de  MOP  est  lies  e  cells  de  mesure  Ainsi,  dans  le  cas  g6n6ral,  on 
regroupera  sous  le  terms  de  MOP  la  donnee  d'une  sequence  d'opdrateurs  bas 
niveau  (eventuellement  un  seul)  et  d'un  ensemble  de  mesures  e  effectuer  a 
posteriori.  Ces  mesures  ont  pour  but  de  juger,  dans  la  mesure  du  possible,  du 
rbsultat  de  I'application  du  MOP  courant,  et  eventuellement  de  guider  le  choix  et  les 
conditions  duplication  des  MOPs  ulterieurs. 


La  generation  dynamique  de  plan 

Le  choix  d'une  filibre  revient  b  seiectionner  une  succession  de  sous-buts  qui 
constituent  autant  d'etapes  dans  la  realisation  du  but  principal  trouver  la 
localisation  d’un  symbols  dans  une  image. 

La  fagon  dont  est  realise  chaque  sous-but  est  determines  dynamiquement  en 
fonction  du  contexts.  En  effet,  soit  ce  sous-but  est  lui  m6me  redbcomposable,  soit  il 
est  realisable  par  une  seule  transition  dans  le  graphs  des  opbrateurs  de  vision. 
Cette  prise  en  cornpte  diffbr6e  des  choix  locaux  d'operateurs  permet  de  tirer  le 
meilleur  profit  des  r6suliats  issus  des  mesures  inteimediaires. 

On  distinguera  done  deux  types  de  MOP  : 

.  les  MOP  terminaux  correspondent  de  manibre  univoque  &  une  sequence 
"indivisible"  d'operateurs  vision  bas  niveau, 

.  les  MOP  non  terminaux  se  decompose,  de  manibre  unique  ou  non,  sous  forme 
d'une  sequence  de  MOP  terminaux  ou  non  terminaux. 


5.  MAQUETTE  DF.  DEMONSTRATION 
REALISATION  -  EXPERIMENTATION 


5.1.  INTRODUCTION 

La  maquette  de  demonstration,  objet  du  marche,  a  represents  un  travail 
considerable  rbalisbe  sur  machine  Lisp  SYMBOLICS  3650  : 

-  plus  de  20000  lignes  Lisp  enticement  onginales, 

-  plus  de  90000  lignes  de  C  dont  moitie  d'onginales  pour  la  librame  de  traitement 
d'image, 

-  saisie,  recalage  et  documentation  de  plus  de  60  scenes, 

-  experimentation  du  systbme  sur  ces  scenes. 


5.2.  EXEMPLE  ^INTERPRETATION  DE  SCENE 


Nous  allons,  dans  cet  example,  interpreter  une  scene  semi-urbaine.  Le  systeme 
commence  par  determiner  les  symboles  visibles  ei  leur  apparence  a  priori.  II  les  trie 
et  ne  retient  que  2  symboles  intSressants  .  le  chateau  d'eau  d'Aimargues  et  un 
trongon  de  route.  II  part  sur  ('identification  du  chateau  d'eau  avec  une  fenetre  de 
recherche  correspondent  S  la  localisation  optimale  (Figure  5.1). 


Aprbs  derivation  du  symbols  courant,  on  effectue  une  phase  de  mesure  sur  Timage 
afin  de  completer  le  modeie  radiometrique. 

On  ordonne  et  trie  les  filibres  pour  n'en  retenir  que  deux  : 

-  une  approche  morphologie  math6matique  /  analyse  de  connexite, 

-  une  approche  projection  de  points  de  contours. 


On  entre  dans  le  module  ContrSle  Vision  avec  la  l4r*  fili6re,  macro-opbrateur  non 
terminal,  qui  se  decompose  en  un  Is'  macro-opbrateur  rbalisant  un  "chapeau  haut 
de  forme"  suivi  d’un  seuillage  adaptatif  pour  mettre  en  evidence  les  zones  blanches 


Le  2ime  macro-opbrateur  non  terminal  est  ensuite  decompose  et  la  branche 
"ouverture  morphologique"  est  retenue  : 
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Le  3«m«  macro-operateur,  terminal,  est  une  analyse  de  connexite  suivi  d'un  tri  qui 
nous  fournit  une  seule  localisation  dans  1‘image  : 


On  remonte  au  module  Identification  qui  lance  en  confirmation  la  24m8  fili&re 
disponible  dont  la  premi&re  operation  est  une  extraction  de  points  de  contours 
verticaux  : 


On  projette  ensuite  et  on  retient  deux  pics  d'une  largeur  correspondant  k  la  largeur 
attendue  du  chateau  d'eau  : 


On  effectue  dans  chaque  tranche  une  projection  horizontale  et  on  recherche  les 
pics  pouvant  correspondre  au  chateau  d'eau  : 


**u*«>*s*MiS*Ja 
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On  remonte  ces  2  localisations  &  ('Identification  qui  va  fusionner  avec  les  rbsultats 
de  la  detection  et  ne  fournir  qu'une  localisation.  On  remonte  cette  localisation 
fusionnbe  au  module  Navigation  qui  61abore  I'hypothbse  de  localisation  Loc-1 
(planche  5  1)  avec  une  estimbe  de  I'erreur  de  3G  ■  -8  m  3L  =  39  m  pour  une  erreur 
introduce  de  50  m  sur  chaque  axe  (indiqu6e  par  le  point  dans  la  len§tre  de 
presentation  de  I'estim6e  d'erreur  de  position).  L'estim6e  de  I’erreur  est  une  ellipse 
allonges  (petit  axe  =  11  m)  dans  la  direction  de  la  ligne  de  vis6e  avion  -  chateau 
d'eau.  C'est  sur  cette  base  que  va  etre  blaboree  la  fen§tre  de  recherche  du  Trongon 
de  route. 


3Cf3  system®  a  base  de  connalssorices  <Talde  a  nril*rp«etot»o<i  dlmages  Infrarooges  a  bord  d’un  vehkicJ* 


CirstunMbM  GL  Edition  Symbol*  Initialisation  S*is«  Ccran  Tra^t  G r«nd*i  Lijr** 

Caracunmion  Symbols  Effactr  Ecran  Inurjettauen  Sawtaard*  CarU  Tra^c 

C*nw*«Uir«*  Effsosr  GL  Iscturs  Carla  Trac«  CarU 

Edition  CL  Cffa*.*r  Symbol*  RAZ  Can*  Tra<«  Eur**xi 


(c<3 

toll  tonnsndl 
toll  'annand 
Sell  .  Onnarwl t 
Sell  connandi 
Sell  iwr*<y) 
&<l)  conn and! 
Set)  oonnand 
Sel)  connand 
So  1 3  tow* nd. 


! 

!  \ 

\ 

y 

Localisation  Optimale 


Localisation  Loc-1 


Localisation  Loc-1 -1 


FIGURE  5.1 

CARTE  ET  GRAPHE  DE  LOCALISATION  -  EVOLUTION  DE  L'ESTIMEE  DE 
L'ERREUR  INERTIELLE  ET  DE  L'ERREUR  D'ESTIMATION 
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6.  CONCLUSIONS  ET  PERSPECTIVES 


Une  realisation  complexe 

Tant  par  les  techniques  qu'il  emploie,  que  par  les  connaissances  mises  en  oeuvre  dans 
les  modules  "intelligents",  le  systems  est  d'une  Ires  grande  richesse.  La  complexite 
resultants  a  entraind  de  lourds  developpements  mais  elle  etait  ndcessaire  pour 
rdsoudre  le  problems  pose.  II  est  mteressant  de  souligner  que  les  techniques  de 
programmation,  fonctionnelle  et  surtout  declarative  qui  sont  une  des  composantes  de 
I'lntelligence  Artificielle  ont  contribu6  e  la  reussite  du  projet  autant  que  les  techniques 
"cogmtives". 

Des  techniques  innovantes 

Dans  chacun  des  domaines  pr6-cites,  des  solutions  interessantes  ont  6te  retenues, 
nous  mettrons  ici  I'accent  sur  deux  d'entre  elles.  La  variabilite  de  I'apparence  tant 
geometrique  que  radiometrique  des  symboles  a  conduit  e  detinir  une  representation 
abstraite  de  leur  apparence  dans  I'image.  Sur  la  base  de  ce  models  il  a  et6  possible  de 
realiser  des  fonctions  de  tri,  comparaison  de  resultats  de  filidres  de  traitement  d'image, 
pourtant  de  natures  trds  ditf6rentes.  Ceci  a  6te  realise,  par  des  mesures  de  distance 
entre  models  et  resultats,  rendues  possibles  gr£ce  £  cette  representation.  La 
representation  des  resultats  de  I’mterpretaticn  sous  forme  d'hypotheses  d'erreurs  de 
position  mertielle  du  porteur,  facilite  leur  exploitation  £  fins  de  confirmation  pour  la 
reconnaissance  d'autres  symboles  que  ce  soit  dans  une  meme  image  ou  £  des  instants 
distincts 

Des  resultats  prometteurs 

Dans  sa  configuration  laboratoire  actuelle,  la  maquette  permet  de  reconnaitre  plusieurs 
types  d'amers  dans  des  images  FLIR  basse  altitude  :  trongons  droits  de  voies  de 
communication,  superstructures  teiles  que  tours,  chateaux  d'eau  et  autres  batiments 
Isolds.  Le  taux  de  bonne  detection  est  £  I'heure  actuelle  do  80%.  L'6chantillon  d’images 
utilise  reste  cependant  limite  par  rapport  £  I'ensemble  des  conditions  que  pourrait 
rencontrer  un  systdme  opdrationnel  et  il  y  a  lieu  d'etre  encore  prudent  sur  ce  point. 

Des  sujets  de  recherche 

La  maquette  r6alis£e,  est  aujourd'hui  un  point  d'appui  pour  des  d6veloppements 
int6ressants : 

Le  mode  Observation,  tr£s  int6ressant  d'un  point  opdrationnel  et  thdorique,  n'a  pas 
encore  dtd  dtudid.  Sa  rdalisation  suppose  I'ajout  de  fonctions  de  poursuites  dans 
i'image  pour  ddtecter  les  vdhicules  et  la  mise  en  place  d'une  interprdtation 
contextuelle  (lieu  et  vitesse  de  ddplacement)  pour  leur  identification 
.  Dans  son  fonctionnement  actuel  le  systdme  pourrait  etre  amdliord  grace  £  des 
capacitds  d'apprentissage  "en  opdration",  notamment  au  niveau  de  la  prdvision  de 
I'apparence  thermique  des  symboles  pour  laquelle  on  pourrait  utiliser  le  rdsultat  de 
reconnaissances  ddj£  rdalisdes. 

Des  applications  potentlelles 

En  plus  d'applications  adroportdes,  le  systdme  pourrait  dtre  utilisd  sur  vdhicule 
terrestre,  autonome  ou  non,  pour  des  applications  de  recalage  et  de  suivi  de  route.  Une 
prise  en  compte  de  nsques  de  masquages  plus  dlevds  et  de  moddles  plus  complexes  au 
niveau  de  I'apparence  des  symboles  dans  I'image,  amenant  des  traitements  d'images 
plus  ddlicats,  seraient  les  prmcipaux  points  £  ddvelopper  pour  ce  type  duplication. 
Enfin,  des  applications  en  exploitation  d'images  satellites  ou  de  reconnaissance 
adrienne  sont  dgalement  envisageables. 
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ABSTRACT 


The  aim  of  this  paper  is  to  describe  AERITALIA’s  (Defense  Aircraft  Group)  experience  with  the 
problem  of  path  generation  and  evaluation  for  a  multitargct  air  to  ground  planning  system  working  in  a  limited 
geographic  scenario. 

Since  in  the  same  planning  operation  we  have  to  satisfy  many  goals  (each  goal  is  a  mission  target)  we 
have  to  use  techniques  of  splitting  problems  into  simpler  ones  as  it  is  not  possible  to  use  satisfactorily  heuristic 
guided  search  (like  A*).  Although  these  algorithms  give  an  optimal  solution  for  each  goal,  this  doesn’t  ensure 
that  the  union  of  the  solutions  that  arc  optimal  will  be  the  best  for  the  whole  set  of  goals. 

The  mission  is  decomposed  into  sub-missions,  and  each  one  of  them  is  solved  finding  all  the  solutions 
satisfying  the  forced  constraints  (c.g.  maximum  leg  length,  maximum  turn  number).  All  the  solutions  so 
generated  become  a  new  search  space  to  be  used  to  find  out  the  final  best  solution. 

Every  sub-mission  is  represented  by  a  task  (a  process)  that  generates  all  the  partial  solutions  (partial 
paths)  that  will  be  combined  in  an  upper  stage  by  a  higher  level  task  (mission  task)  in  order  to  generate  the 
whole  set  of  global  solutions. 

Currently  all  the  tasks  arc  independent  and  arc  managed  by  a  task-scheduler  which  simulates  the 
behaviour  of  a  multiprocessor  architecture  carrying  out  the  task  synchroniza'ion  by  an  event  generation  and 
waiting  mechanism.  A  few  simple  primitives  allow  the  tasks  to  generate  events  and  wait  for  them  (c.g.  waiting 
for  the  end  of  a  task  computation). 

Every  sub-mission  task  builds  each  path  as  a  collection  of  atomic  paths  (point-to-point)  which  may  be 
shared  by  different  solutions  of  the  task.  The  mission  tasks  will  combine  the  subtosks  results,  generating  a 
graph  whose  roots  are  all  possible  paths  and  whose  leaves  are  the  atomic  paths  (a  leg).  Afterwards  the  best  ones 
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among  these  paths  arc  chosen  using  a  bayesian  evaluation  system  that  traverses  the  path  graph  calculating  the 
main  features  (c.g  path  length,  dangerousness  etc.)  and  sending  the  values  calculated  in  the  leaves  to  the  roots. 

The  evaluator  is  a  graph  which  has  path  feature  evaluation  nodes  and  bayesian  combination  nodes 
whose  function  is  to  combine  evaluations  in  one  single  value.  The  criteria  used  concern  risk  exposition,  path 
rccogni/ability,  consumption,  altitude  profile,  turn  width,  success  probabilty  on  target.  The  evaluation  graph  is 
run  time  configurable  to  make  possible  the  use  of  all  the  criteria  or  any  subset. 

The  developed  prototype  has  been  tested  by  our  pilots  and  the  results  arc  described  below. 

The  system  has  been  developed  on  a  lisp  machine  Explorer  11  using  Lisp  and  Flavors  and  later  |>orted 
on  a  Symbolics  3630. 


*■ 
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Introduction:  The  planning  problem 


The  generation  of  a  flight  plan  for  a  ground  attack  mission  in  visual  flight  is  a  process  that  finds  a  path 
joining  a  sequence  of  way  points  which  connect  the  take  off  airfield  with  the  landing  one  through  the  planned 
targets  of  the  mission. 

A  mission  planner  t'-sed  in  the  field  must  be  able  to  generate  for  each  mission  an  optimal  flight  plan  (or 
a  set  of  plans)  with  respect  of  a  certain  set  of  criteria.  Pilots  must  always  have  the  possibility  to  choose  their 
flight  plan  from  a  set  of  plans  (in  order  to  avoid  always  having  the  same  plan  for  the  same  mission).  Besides  the 
solutions,  the  mission  planner  must  provide  information  about  the  probability  of  success  of  the  mission,  risk  of 
the  route,  consumption,  etc. 

A  set  of  constraints  must  be  taken  in  account  to  build  the  paths  that  will  become  flight  plans.  These 
constraints  will  be  used  in  the  generations  of  plans  together  with  a  set  of  criteria  for  evaluating  the  quality  of  the 
plans  thomsclr.  Many  of  these  criteria  and  constraints  are  based  on  pilots  experience.  When  pilots  have  to  plan  a 
mission  they  use  theirflight  experience  and  knowledge  of  the  territory  they  must  flight  over.  They  also  use  their 
knowledge  of  the  aircraft  performance,  personal  strategics  and  attack  modalities. 

Nowadays  the  task  of  preparing  flight  plans  is  performed  by  pilots  who  arc  the  best  experts  in  this  field 
(  and  obviously  the  most  direct  interested  in  the  plan's  quality).  An  automatic  planner  must  generate  plans  at 
least  as  good  as  those  of  pilots  (better  if  possible)  so  it  must  take  into  consideration  the  same  knowledge  that 
pilots  use  in  the  problem  resolution  and  that  they  have  acquired  during  their  flight  experience. 


The  path  generation  and  evaluation:  an  analysis  from  the  knowledge  point  of  view. 


A  detailed  analysis  of  the  planning  problem  (if  not  exhaustive)  is  necessary  to  better  understand  the 
problems  one  can  meet  in  the  development  of  an  automatic  mission  planner.  This  analysis  has  been  possible 
only  observing  how  a  pilot  plan  a  mission.  Between  the  different  elicitation  techniques  the  most  useful  was  to 
ask  pilots  to  think  aloud  during  a  scries  of  sessions.  The  records  of  these  sessions  allowdcd  a  better 
understanding  of  the  way  in  which  pilots  work  and  to  gather  tire  first  kernel  of  knowledge,  later  gradually 
uicrcascd.  Even  some  flights  were  useful  to  effectively  understand  the  pilots  point  of  view.  During  these  flights 
pilots  explained  to  us  in  the  field  their  problems,  the  knowledge  involved  and  its  use. 

In  the  introduction  we  asserted  that  to  plan  a  mission  means  to  determine  a  path  joining  a  set  of  way 
points  so  as  to  connect  the  take  off  airfield  with  the  landing  one  through  the  defined  targets  of  the  mission. 

The  way  points  to  be  connected  must  satisfy  a  set  of  requirements  that  restrict  the  choice  possibilities  in 
the  definition  of  the  paths.  These  requirements  are:  the  probability  of  a  correct  identification  of  way  points 
during  flight  according  to  the  arrival  direction  and  the  constraint  to  obtain  paths  in  certain  lenght  range  with 
limited  and  not  too  frequent  turn  angles  (geometrical  route  constraints). 
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Moreover  it  is  also  necessary  to  take  account  of  the  orographic  information  of  the  territory  to  be  flown 
over  and  of  the  possibility  of  finding  threats  on  the  route.  Thus  it  is  necessary  to  find  an  optimal  path  according 
to  different  points  of  view  (length,  threatness,  probability  of  correct  identification  of  the  way  points,  etc.),  some 
of  them  can  contrast  each  other. 

Some  evaluation  criteria  depend  on  the  average  value  of  certain  characteristics  of  the  calculated  path 
(c.g.  the  identification  of  the  way  point),  others  depend  on  lire  complete  value  of  the  path  (lenght,  threatness 
etc.).  These  evaluation  criteria  are  calculated  in  different  ways:  this  fact  influences  the  generation  of  the  route.  In 
fact  it  means  that  we  cannot  use  heuristic  search  algorithms  in  the  space  of  the  states  (i.c.  algorithm  of  the  A* 
class  (6))  because  they  require  incremental  evaluation  functions  in  order  to  be  used  with  a  certain  advantage. 

Moreover  it  is  also  necessary  to  take  into  account  tactical  criteria  particularly  in  choosing  the  optimal 
push  up  point  on  the  target.  This  kind  of  choice  depends  especially  on  the  target  type  (bridge,  highway  junction 
etc.),  on  the  armaments  and  on  the  attack  type  (toss,  lay  down,  etc.).  Finally,  the  flight  plan  must  take  into 
account  the  temporal  constraints  (i.c.  time  on  target). 

So  the  problem  to  build  a  flight  plan  for  a  mission  becomes  the  problem  of  defining  paths  linking  the 
focal  points  of  the  mission  (take  off,  targets,  landing)  so  as  to  satisfy  some  constraints  and  to  be  optimal 
according  to  certain  requirements. 

At  the  first  glance  one  could  think  to  reduce  the  problem  into  subproblcms  (c.g.  from  the  take  off  base 
to  the  first  target,  from  this  last  to  tire  next  and  so  on  till  the  landing  base)  and  then  to  resolve  separately  these 
subproblcms  and  unify  the  results  to  find  out  the  complete  solutions. 

In  general  this  is  not  possible  because  constraints  and  optimality  requirements  arc  used  in  every  point 
of  the  route,  so  it  is  not  possible  to  guarantee  that  they  are  respected  in  the  joining  points  of  the  subproblcm 
solutions. 

In  other  words,  in  our  ease  the  simple  "sum"  of  local  optimal  solution  doesn't  guarantee  an  overal 
optimality.  In  some  eases  this  is  possible,  it  depends  on  the  mission  characteristics.  For  instance,  a  reiterated 
attack  allows  to  leave  the  target  from  any  direction  so  the  geometrical  constraints  on  the  route  falls  down  and  it 
is  possible  to  obtain  a  global  optimum. 


The  developed  prototype 


The  developed  prototype  (currently  it  works  on  a  Symbolics  3630  and  is  implemented  in  Common  Lisp) 
is  composed  of  four  main  modules. 
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-  an  object  oriented  data  base  containing  all  the  necessary  information  to  represent  the  territory 

(orography,  way  points,  towns,  roads,  rivers  and  so  on)  and  the  tactical  scenario  (missiles,  artillery, 
radars  with  their  operative  range).  Furthermore  the  data  base  contains  heuristic  information:  every 
way  point  has  an  identifiabllity  diagram  (given  by  pilots)  and  targets  have  attack  diagrams  pointing 
out  goodness  factor  of  the  arrival  direction  on  the  target  according  to  orography,  attack  type,  target 
type.  For  each  class  of  target  (bridge,  airfield  etc.)  there  is  an  attack  diagram  drawn  from  the 
information  given  by  pilots.  These  diagrams  arc  instantiated  to  the  specific  eases  using  functions 
that  take  into  account  the  local  data  (orientation,  position  etc.). 

-  a  module  exaustively  generating  all  the  permissible  paths.  This  module  uses  an  algorithm  that  combines 

the  reducing  problem  into  subproblcm  techniques  with  a  breadth  first  algorithm  on  a  graph.  The 
problem  is  decomposed  in  tasks  (from  take  off  to  the  first  target ...  from  the  last  target  to  landing): 
tor  each  task  every  path  satisfying  the  geometrical  constraints  on  route  (maximum  number  of  turns, 
maximum  turn  angle,  etc.)  is  found,  in  the  end  the  task  Dial  generates  the  complete  paths  combines 
ail  the  partial  paths  (given  by  the  previous  tasks)  removing  the  solutions  that  doesn’t  verify  the 
constraints  in  the  conjunction  points.  The  final  solutions  arc  then  evaluated  and  ranked  so  to  be  able 
to  choose  the  best  ones. 

Obviously  such  an  algorithm  doesn't  have  real  time  performance,  but  we  had  different  reason  in 
choosing  this  approach  for  the  time  being. 

First:  an  analysis  of  the  space  of  the  problem  states  space  guaranteed  that  although  it  is  computationally 
demanding,  the  algorithm  was  executable  in  acceptable  time  and  that  its  complexity  (exponential  according  to 
the  way  point  number  of  the  route)  had,  in  our  ease,  a  finite  and  computable  maximum  limit. 

Secou  1:  this  algorithm  becomes  a  workbench  to  gather,  test  and  evaluate  the  necessary  knowledge  to 
evaluate  tl,«.  paths  quality.  In  fact,  having  all  the  acceptable  solutions,  it  is  possible  to  verify  the  better  ones, 
together  with  the  rejected  ones:  in  this  way  it  is  possible  to  verify  with  the  experts  if  any  potential  good  solution 
has  been  rejected  and  to  controll  the  quality  of  the  knowledge  introduced  in  the  system. 

Finally,  this  is  the  basis  on  which  it  is  possible  to  develop  other  algorithms,  not  exaustivc  and  almost 
real  time,  testing  the  quality  of  the  found  solutions  and  the  difference  from  the  theoretic  optimal  solution. 

-  an  evaluation  module  based  on  a  bayesian  evaluation  network  rcconfigurablc  to  work  with  different 

criteria  sets. 

-  a  batch  oxcution  and  evaluation  module  that  runs  a  set  of  benchmark  tests.  This  module  allow  to  to 

execute  benchmark  to  evaluate  different  algorithms  from  the  point  of  view  of  the  solutions  quality. 

These  four  module  form  a  "dedicated  shell"  in  which  it  is  possible  to  study  and  to  evaluate  the  best 
approach  to  the  problem  and  on  which  it  is  possible  to  insert  different  planning  algorithms  and  to  evaluate  their 
quality  "or  the  field". 


Below  there  arc  some  consideration  got  from  the  first  us<  ■,  of  this  shell. 
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Search  algorithm  to  generate  plans 


The  spectrum  of  scatch  algorithm  extends  from  brute  force  techniques,  which  use  no  knowledge  of  the 
problem  domain,  to  knowledge  intensive  heuristic  search! 

The  problem  of  the  brute  force  algorithms  is  their  complexity  that  cat.  lead  to  unacceptable  execution 
time  if  the  state  spaces  is  not  well  limited.  Their  strength  is  that  they  guarantee  not  only  to  find  always  the  best 
solution  but  also  to  find  other  solutions,  if  there  are,  good  enough  to  be  used.  In  this  case  pilots  have  the  dtoice. 

The  heuristic  approach  uses  domain  knowledge  intensively  to  guide  the  search.  This  may  be  in  the 
form  of  cost  functions  and  of  estimating  functions  if  a  search  algorithm  is  used  (A*  (6),  bidirectional  search  US* 
(4)  etc),  or  in  the  form  of  strategies  if  more  complex  planning  tccniques  arc  used. 

In  our  ease,  search  algorithm  based  on  heuristics  showed  some  problems:  first  of  all  some  cost  functions 
are  not  monotone  so  optimality  of  the  solution  is  not  guaranteed  (in  our  experiments  with  a  bidirectional  search 
algorithm  US*,  about  30%  of  the  best  solutions  were  not  really  the  best  solutions  compared  with  the  hexaustivc 
approach).  Secondly  these  kind  of  algorithm  v/erc  ‘■tudied  to  compute  mainly  the  best  solution,  not  a  set  of  the 
best  solutions.  Moreover,  even  if  wo  modify  these  algorithms  to  obtain  the  n  best  soluUons  we  are  not 
guaranteed  that  these  solutions  are  effectively  different  each  other,  and  pilots  consider  very  important  the 
possibility  to  choose  between  a  set  of  best  solutions.  In  most  eases  these  algorithms  find  very  similar  solutions 
(c.g.  they  differ  only  for  a  way  point  sometimes  near  to  those  of  the  other  route)  and  don't  allow  a  significant 
choice.  Obviously  the  exaustivc  algorithm  too  suffer  from  the  same  problem,  but  having  all  the  acceptable 
solution,  it  is  possible  the  search  of  a  different  path. 

Finally,  most  of  the  classic  search  algorithms  are  optimal  to  find  the  best  path  between  two  points,  but 
don't  consider  the  case  in  which  one  has  to  connect  any  number  of  points:  so  they  must  be  used  together  with 
reduction  problem  techniques.  Furthermore,  this  is  not  enough,  in  fact  even  in  this  way  we  are  not  guaranteed 
to  find  the  best  solution,  so  we  must  also  use  more  sophisticated  mechanism  to  allow  reasoning  tccniques  as  the 
"hypothesize  and  test". 

The  development  of  a  more  sophisticated  planning  system  that  use  knowledge  to  generate  strategies  to 
plan  seems  promising.  The  fundamental  point  here  is  the  way  in  which  the  system  "sees"  the  map.  In  fact  pilots 
arc  able  to  read  the  map  globally  and  to  extract  elements  (valley  or  hills  to  hide  to  radars)  useful  to  generate 
problem  solving  strategics:  in  other  words  pilots  find  out  “macro  paths"  that  try  to  refine  respecting  constraints. 
Contrarily  classic  heuristic  search  algorithm  arc  able  to  read  the  map  locally,  near  the  point  they  arc  expanding: 
so  it  is  not  possible  to  extract  global  strategies.  Performance  and  flexibility  of  the  first  approach  are  remarkable: 
for  instance  it  is  possible  to  use  different  strategics  in  different  situations.  1  Iowcvcr  this  is  an  approach  still  in 
tire  thcoric  phase  and  it  requires  a  very  strict  interaction  with  pilots  and  cxpccially  an  interdisciplinary 
approach  in  the  development  phase:  for  instance  a  module  to  "read  the  map  globally"  may  require  image 
interpretation  techniques. 
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The  system  rule  in  the  knowledge  elicitation 


The  use  of  the  system  as  a  task  oriented  shell  was  particularly  useful  in  knowledge  acquisition,  allowing 
a  dialog  between  system  and  expert  who  is  able  to  evaluate  the  difference  between  his  solutions  and  those 
offered  by  the  system.  In  this  way  wo  obtained  an  effective  reflection  on  knowledge,  more  and  more  useful  in 
respect  to  that  obtained  in  theory. 

Pilots  interaction  with  the  system  is  made  by  a  classic  multi-window  interface,  using  mouse  and  menus. 
The  interface  allow  a  synthetic  map  representation  on  which  way  points,  cities,  rivers,  highways  etc.  and  the 
tactical  scenario  (feba  and  threats)  are  displayed  (see  fig.  1  at  the  end  of  the  paper).  It  is  possible  to  visualize  on 
this  map  the  different  solutions  (fig.  2):  in  this  way  pilot  has  an  output  similar  to  that  commonly  used.  The 
interface  allows  some  functions  to  edit  the  tactical  scenario  (create  and  remove  of  threats,  feba  definition),  to 
introduce  all  the  necessary  data  to  define  the  mission  (take  off,  landing,  targets,  weapons,  attacks  etc)  and  to 
introduce  the  pilot's  own  solution.  This  last  is  a  very  useful  option  as  it  allows  to  verify  the  knowledge  used  in 
the  evaluation  of  the  system  solutions  directly  on  the  pilot  one,  verifying  immediately  its  quality. 

Tire  possibility  to  modify  quite  quickly  parts  of  the  bayesian  network  used  for  the  evaluation  (e.g. 
changing  ccrtanty  factors)  allows  a  tight  pilot-system  interaction  and  makes  knowledge  refinements  easier. 

The  direct  interaction  expert-system,  helped  by  a  knowledge  engineer  who  may  (when  possible)  correct 
the  system,  allows  the  expert  to  get  involved  much  more  in  the  system  development  and  in  the  work  team.  In 
fact  in  this  way  of  work,  the  expert  can  see  his  words  to  become  quickly  part  of  the  system. 

On  the  other  hand,  there  arc  some  unfavourable  aspects  in  this  approach:  for  instance  there  is  the  risk  to 
focus  the  attention  on  the  interface  development  (easy  use,  good  display,  etc.)  instead  of  working  effectively  to 
gather  knowledge  This  is  a  real  risk  because  the  time  required  to  develop  a  good  interface  is  remarkable,  its 
complexity  loads  on  the  whole  system  performance  (e.g  page  faults)  and  this  process  tends  spiral. 


The  qualitative  tests  of  the  prototype  and  their  evaluation 


Tests  were  executed  on  a  limited  geographic  scenario  (about  200  x  250  km)  using  the  cxausdvc 
algorithm.  The  tests  objective  was  to  verify  the  correctness  and  consistency  of  the  data  necessary  to  generate  the 
solutions  and  to  control  their  acceptability  and  quality.  As  we  were  not  interested  in  verifing  the  algorithm 
performance,  the  algorithm  used  (clearly  not  real  time)  and  the  limited  scenario  didn't  affect  the  tests  results. 

Tests  followed  the  modalities  described  below: 

-  Definition,  together  with  pilots,  of  three  tactical  scenario  (one  only  with  the  feba,  the  others  with  SAM-6 

and  SAM-7  missiles  and  anti  aircraft  artillery) 

-  Definition,  together  with  pilots,  of  a  meaningful  test  set  in  order  to  cover  the  different  possibilities  (take 

off,  landing,  different  type  of  targets,  different  number  and  kind  of  attack)  The  test  set  was  applied 

to  the  three  tactical  scenario. 
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-  Pilots  were  given  a  map  with  the  same  information  stored  in  the  system,  in  order  to  work  in  the  same 

situation  as  the  system  (pilots  uses  a  lot  of  information  to  plan  their  mission,  some  of  them  as  the 
power  transmission  lines  are  drawn  on  the  aeronautical  chart  but  aren't  stored  in  our  system). 

•  Pilots  were  asked  to  plan  the  mission  of  the  test  set  alone.  At  the  same  lime  the  system  planned  tire  same 
missions. 

-  Pilots  were  asked  to  evaluate  the  best  solutions  proposed  by  the  system  and  the  system  evaluated  the 

solutions  proposed  by  pilots.  The  system  works  in  an  exaustive  way  finding  every  possible  solution 
and  choosing  the  best  ones:  so  from  now  on  when  we  say  "different  solutions”  proposed  by  pilots 
we  intend  "different  from  the  best  solutions  proposed  by  the  system". 

Sometimes  pilots  found  different  solutions,  in  spite  of  the  exaustive  way  of  work  of  the  system.  This  is 
because  the  system  finds  every  possible  route  satisfying  the  geometrical  constraints,  pilots  ins'ead  may  find 
solution  lightly  out  of  the  admissible  ranges.  We  developed  a  mod,,lc  for  the  introduction  of  ca'culate  solutions 
specially  for  this  cases,  in  order  to  undergo  them  to  the  system  evaluation. 

-  Analysis  of  the  results.  We  consider  the  quality  level  given  by  the  pilots  and  by  the  system,  and  we 

extract  the  points  of  the  routes  meaningful  different.  In  fact  sometimes  there  were  equivalent  points 
near  each  other  so  in  this  case  the  choice  and  therefore  the  difference  in  the  route  wasn't  important. 

It  was  particularly  important  to  evaluate  the  difference  in  the  choice  of  the  push  up  and  escape 
point.  These  last  kinds  of  difference  involve  different  approach  to  the  target  and  therefore  an 
inconsistency  between  the  tactical  knowledge  used  by  pilots  and  that  of  the  system. 

From  the  tests  analysis  it  results  that  pilots  often  gave  different  solutions  to  each  other.  One  of  the  pilots 
who  made  the  tests  was  the  leading  expert  for  the  whole  project  development:  as  we  thought  the  system  found 
solutions  nearer  to  the  leading  expert  ones.  This  shows  the  subjectivity  of  part  of  the  system  knowledge 
necessary  for  this  kind  of  work,  but  this  is  also  a  demonstration  of  the  flexibility  of  the  artificial  intelligence 
techniques  in  representing  and  using  not  well  formalized  and  highly  subjective  knowledge. 

In  any  ease,  as  regards  the  quality  of  the  solutions,  no  one  was  substantially  different  from  the  solution 
proposed  by  pilots.  Pilots  evaluated  some  of  the  system  solutions  better  than  theirs  and  didn't  consider  any 
system  solution  decidedly  inferior. 

A  more  detailed  analysis  showed  that  40%  of  the  push  up  points  and  50%  of  the  escape  points 
corresponded  with  those  of  the  reference  pilot,  while  the  proportion  became  lower  with  other  pilots.  On 
average  the  increment  of  the  number  of  targets  corresponds  to  an  increment  of  the  difference.  Sometimes  the 
system  revealed  a  lack  of  fuel,  but  repeating  the  same  mission  with  more  fuel,  the  system  found  meaningful 
better  solutions.  In  the  eases  in  which  the  push  up  points  in  the  system  and  pilots  solutions  were  different,  the 
respective  single  evaluation  (of  the  push  up  point)  was  equivalent  (see  fig.  3  at  the  end  of  the  paper). 

As  regard  the  quality  of  the  overall  solutions,  the  opinion  was  good.  There  was  only  one  critique:  the 
system  often  passes  nearer  the  throats  than  pilots  do.  Pilots  prefer  to  avoid  threats,  passing  far  from  them  even 
if  they  are  sure  of  threats  location:  especially  when  weather  is  not  good  and  visibility  is  poor  it  becomes  very 
important  to  be  sure  to  got  out  of  the  operative  range  of  the  threat.  Some  of  the  differences  were  due  to  this 
consideration,  In  any  ease  this  is  an  Information  about  the  necessity  to  proceed  to  another  and  more  intense 
phase  of  knowledge  acquisition. 
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In  general  the  main  needs  verified  during  the  execution  of  the  test  set  were  of  three  kind: 

-  the  need  to  improve  the  geographic  data  base  (both  in  extension  and  resolution)  in  order  to  have  a  better 

agreement  with  the  operative  use  of  the  system. 

-  the  need  to  give  more  flexibility  to  the  system  in  order  to  release  part  of  the  constraints  when  it  is 

necessary 

-  the  need  to  continue  the  process  of  knowledge  acquisition  about  tactical  aspects  in  order  to  cover  a 
greater  number  of  attack  modalities  and  to  consider  a  greater  number  of  type  of  target. 


Future  works. 


Future  works  will  carry  a  transformation  of  the  prototype  from  a  dedicated  shell  to  a  power  and 
engineered  version  to  be  used  on  the  field. 

The  two  main  items  will  be: 

-  the  tuning  and  engineering  of  all  the  modules  that  were  developed  with  essential  functionalities  as  they 

were  necessary  but  not  crucial  to  the  system. 

-  the  development  of  the  final  planning  module,  after  the  test  of  other  search  algorithms  and  more  and 

new  refined  planning  Iccniqucs. 

Amongst  the  first  kind  of  modules,  the  geographic  data  base  will  be  one  of  the  more  interested  by  the 
growth:  it  will  be  able  to  treat  more  and  more  data  both  because  we  will  enlarge  the  territory  and  because  the 
information  will  have  a  greater  resolution  level.  Moreover,  it  will  be  necessary  to  develop  the  kernel  of  a  DBMS 
(data  base  management  system)  in  order  to  manage  data  stored  on  files,  reducing  the  use  of  the  virtual  memory 
(actually  widely  used)  and  optimizing  the  response  time. 

As  regard  the  second  point,  we  will  gather  and  organize  new  and  deeper  knowledge  especially  about 
the  tactical  aspects  (attack  modalities  according  to  the  diffrent  type  of  target  and  of  weapons,  run  in  direction 
selection  etc.),  (t  will  be  very  important  to  determine  the  main  strategics  that  pilots  uses  in  the  generation  of 
flight  plans  and  tha  way  in  which  the  system  can  "read  the  map  from  a  global  point  of  view". 

Tire  results  obtained  lead  us  to  consider  useful  the  followed  approach  to  the  development  of  a  ground 
station  to  plan  the  operative  mission. 


p?fi**3*  -W*' XVJJ  V 
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Fig.  2  .  The  line  crossing  the  entire  territory  (right  upper  window)  represents  the  FEBA,  the  right  side  of 
the  FEIJA  is  the  enemy  one.  The  brown  zones  in  the  big  window  arc  dangerous  because  beyond  the  FEBA. 
Moreover,  a  threat  is  set  in  right  lower  part  of  the  window;  the  dangerous  zones  included  in  the  operative  range 
of  the  missiic  ate  orange.  In  this  photo  two  different  routes  are  displayed:  with  the  threat  the  best  route  is  the 
pink  one  (the  lower  route),  without  the  threat  the  best  is  the  black  one. 
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Fig.  3  .  In  this  photo  the  tactical  evaluation  of  a  push  up  point  is  dislaycd.  The  push  up  point  is  the 
railway  junction  of  Santhia  (rwj.santhia)  and  the  target  is  the  railway  bridge  of  Vcrcclli  (rwb.vcrcclU).  The 
overall  evaluation  is  shown  in  the  Information  Display  window  and  depends  upon  the  push  up  rccognizability 
and  upon  the  probability  of  success  on  target.  Moreover  this  last  depends  on  other  factors:  one  is  the  goodness 
of  the  arrival  direction  on  target  that  depends  on  the  target  type  and  on  the  orography  near  the  target  (see 
central  window  tgt-type  and  orografic) 
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SUMMARY 

A  knowledge-based  cockpit  assistant  for  IFR  (Instrument  Flight  Rules)  operation  is  presented,  aimed  at  improvement  of  sit¬ 
uation  assessment  and  performance  increase  by  computer  aids  for  flight  planning  and  plan  execution.  Here,  situation  assess¬ 
ment  also  includes  monitoring  of  the  pilot’s  own  activities.  The  modular  system  structure  is  described  as  well  as  the  individual 
system  modules.  The  cockpit  assistant  was  tested  in  a  flight  simulator  by  professional  pilots  under  realistic  IFR-scenarios.  The 
concept  of  the  test  design  as  well  as  test  results  are  presented.  The  system  design  goals  are  mainly  confirmed  by  these  results. 


1.  INTRODUCTION 

IFR  operation  is  a  standard  for  commercial  air  traffic  m  order  to  warrant  arrivals  on  schedule  under  adverse  weather  condi¬ 
tions.  This  is  also  true  for  a  substantial  portion  of  the  general  aviation  traffic.  However,  lack  of  visual  cues  from  outside  the 
airplane  as  well  as  increased  automation  and  complexity  of  cockpit  instrumentation  can  result  in  overdemanding  loads  on  the 
pilot.  This  leads  to  a  significantly  higher  number  of  accidents  in  IFR  operation,  in  particular  for  single-pilot-IFR  flights 
[1,2,3]. 

Investigations  on  the  cognitive  behavior  of  humans  [4,5]  have  revealed  that  electronic  cockpit  assistance  for  the  pilot  has  a 
good  chance  of  becoming  effective  for 

-  situation  assessment 

-  planning  and  decision  making  and 

-  plan  execution 

by  complementing  human  capabilities  and  not  taking  away  from  him  all  challenges. 

On  the  basis  of  this  formal  body  of  knowledge  of  the  user  needs  a  cockpit  assistant,  called  ASPIO  (assistant  for  single-pilot 
IFR  operation),  has  been  developed  and  implemented  in  a  flight  simulation  facility,  which  is  described  in  the  following. 

An  extensive  test  program  has  been  conducted  in  order  to  provide  data  of  how  performance  improvement,  workload  balanc¬ 
ing  and  pilot  acceptance  can  be  achieved  by  the  aforementioned  cockpit  assistant  functions. 


2,  STRUCTURE  OF  COCKPIT  ASSISTANT 

Conceptually,  from  the  very  beginning  of  the  development,  considerable  emphasis  has  been  put  on  taking  into  account  the 
interdependence  effects  between  the  cockpit  assistant ,  the  pilot,  tbe  ownship  systems  as  well  as  the  air  traffic  and  atmos¬ 
pheric  environment,  looking  for  the  single-pilot  application  in  the  first  place.  Consequently,  the  cockpit  assistant  has  got  three 
main  interfaces  with  its  environment,  i.e.  with  ATC  (Air  Traffic  Control),  with  the  pilot  and  with  the  other  onboard  systems 
of  the  aircraft  (fig.l). 

With  regard  to  the  ATC  interface,  a  digital  two  way  data  link  is  posited.  This  assumption  results  in  ATC  instructions  being 
directly  fed  into  the  assistant  system  and,  at  the  same  time,  being  aurally  presented  to  the  pilot. 

The  pilot  interface  makes  excessive  use  of  speech  communication  in  either  direction.  A  speaker  dependent  speech  recogni¬ 
tion  system  in  single-word  mode  was  used  for  pilot  messages  towards  the  assistant,  rhe  vocabulary  of  about  140  words  and 
word  strings,  in  accordance  to  civil  aviation  terminology,  covers  the 

-  terminology  used  by  the  pilot  for  flight  guidance  management  (e.g.  descend  to  flight  level ...) 

-  command  inputs  towards  the  cockpit  assistant 

-  command  inputs  into  other  aircraft  system* 

-  numbers 

-  instantiations  for  airfieids  and  navigational  aids. 

A  syntax  structure  of  several  levels  is  implemented  to  ensure  sufficient  reliability  and  promptness  of  recognition  (fig.2).  Syn¬ 
thetic  speech  is  used  for  speech  output,  with  different  voices  for  different  categories  of  assistant  messages. 

More  complex  information  like  comprehensive  flight  plan  recommendations  is  presented  visually. 

The  aircraft  interface  is  established  by  a  data  pool  which  contains  all  aircraft  relevant  data  about 
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-  flight  status  (including  control  variables  such  as  autopilot  settings) 

-  radio  navigation  settings 

-  radio  communication  settings 

-  status  of  aircraft  subsystems  (e.g.  engines). 

There  are  three  main  functional  blocks  according  to  the  specified  functions  for  planning  (including  situation  assessment), 
plan  execution  and  monitoring.  The  planning  function  is  provided  by  the  automatic  flight  planner  (AFP)  module.  The  plan 
execution  function  is  broken  down  into  submodules,  i.e.  the  model  of  the  pilot-dying  (MPF),  the  automatic  pilot-not-dying 
(AFNF),  which  is  more  or  less  comprising  what  is  to  be  executed  by  the  co-pilot  in  a  two-man  cockpit,  and  the  autopilot  (AP). 
These  modules,  together  with  the  i»-  "lie  for  plan  execution  monitoring  (MON),  are  described  in  the  following. 


Automatic  Flight  Planner  (AFP) 

For  every  flight,  a  flight  plan  (including  destination)  must  be  issued  before  take  off.  This  flight  plan  could  be  worked  out  by 
the  AFP,  but  it  will  usually  be  prepared  by  means  of  other  facilities  and  will  be  taken  for  the  AFP  initialisation. 

On  the  basis  of  the  actual  flight  plan,  the  AFP  performs  a  rigorous  evaluation  of  the  elements  of  the  current  situation  and  its 
future  projection,  which  might  reveal  that  the  actual  plan  is  no  longer  executable.  Causes  could  be  actual  or  anticipated  de¬ 
viations  from  the  flight  plan  and  such  events  as  new  ATC  instructions  not  in  accordance  with  the  flight  plan,  adverse  weather 
conditions  etc.  For  instance,  this  evaluation  will  activate  the  planning  function  by  means  of  a  problem  solving  algorithm  for 
the  selection  of  an  alternate  destination  and  corresponding  generation  of  a  new  flight  plan. 

For  the  representation  of  the  necessary  AFP  knowledge  in  the  IFR  domain,  the  method  of  problem  reduction  is  used  (fig.3). 
This  representation  can  be  used  for  both  the  situation  evaluation  and  the  planning  itself.  For  some  of  the  planning  decisions, 
fuzzy  criteria  are  used  following  the  theoretical  foundations  of  fuzzy  sets  in  [6], 

The  AFP  planning  results  are  presented  to  the  pilot  as  recommendations,  and  if  not  corrected  by  the  pilot,  these  planning 
results  replace  farmer  flight  plan  commands  for  plan  execution. 


Model  of  pilot-flying  (MPF) 

On  the  basis  of  the  flight  plan  as  generated  by  the  AFP  and  acknowledged  by  the  pilot,  the  MPF  can  perform  automatic  man¬ 
agement  of  flight  plan  execution.  The  rather  coarse  command  structure  of  the  flight  plan  of  the  AFP  rs  broken  down  by  the 
MPF  into  action  sequencies  like 

-  speed  reduction 

-  flaps  actuation 

-  gear  down 
•  final  check 

-  etc. 

for  the  flight  phase  of  the  final  approach. 

These  sequencies  follow  the  regulations  of  piloting  and  ATC,  which  are  extensively  proceduralized.  These  procedures,  being 
automatically  performed  by  the  MPF,  cannot  be  different  from  what  the  pilot  is  supposed  to  put  into  effect.  Therefore,  this 
module  can  also  be  construed  as  a  model  of  the  pilot,  of  which  the  output  can  be  utilized  for  pilot  monitoring  purposes,  too. 

The  MPF  module  can  be  considered  as  almost  exclusively  rule-based.  This  rule  base  is  essentially  structured  by  the  goal/task 
hierarchy  of  the  overall  flight  task.  The  tasks  (or  respective  rules)  are  either  pertinent  to  flight  phases  or  are  dependent  on 
event  occurrences,  represented  by  scripts  and  productions,  respectively. 


Automatic  pilot-not-flying  (APNF) 

The  APNF  represents  the  effector  module  of  the  cockpit  assistant  together  with  the  autopilot  (AP)  module.  This  module 
comprises  all  functions  usually  performed  by  the  co-pilot  in  the  conventional  two-man  cockpit.  Among  these  functions  are 
instrument  setting  flap  and  gear  setting,  ATC  communication,  checklist  execution  and  monitoring  (the  latter  is  directly  passed 
on  to  the  monitoring  module  MON).  There  are  also  standard  callouts  as  usually  delivered  by  the  co-pilot  in  the  conventional 
two-man  cockpit.  The  APNF  performs  these  callouts  via  speech  messages.  The  APNF  can  also  be  directly  tasked  by  the  pilot 
with  respect  to  navigational  calculations  or  requests  about  flight-relevant  information  (radio  stations,  alternates,  etc.). 

There  are  two  modes  of  tasking  the  APNF.  This  is  done  either  by  the  pilot  or  by  the  cockpit  assistant  itself,  depending  on 
where  the  responsibility  of  flying  the  aircraft  is  placed.  The  pilot  may  hand  over  this  responsibility  to  the  cockpit  assistant  in 
the  same  way  as  he  can  pass  it  on  to  the  co-pilot  in  a  conventional  two-man  cockpit.  In  this  case,  the  MPF  module  will  accept 
the  role  of  tasking  the  APNF  and  will  make  use  of  the  effecting  capabilities  of  the  APNF  module  as  well  as  of  the  AP,  as 
described  above.  The  APNF  also  sets  the  AP  modes  and  the  command  values.  If  the  pilot  has  control,  the  AP  may  or  may 
not  be  engaged  accoraing  to  the  pilot  ptefeicnce. 


Monitoring  Module  (MON) 

This  module  works  on  the  basis  of  the  actual  outputs  of  the  pilot,  i.e.  the  resulting  flight  status,  and  the  MPF  outputs.  Devia¬ 
tions  from  the  flight  plan  can  be  detected  by  comparing  these  data  during  flight .  It  thereby  comprises  an  important  segment 
of  the  situation  assessment  task.  This  module  distinguishes  three  categories  of  monitoring  functions: 
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-  Monitoring  of  flight  progress  ('nodal  attention  checks’)  in  order  to  be  aware  of  the  arrival  at  any  subgoals  of  the  flight 
plan 

-  Monitoring  of  health  status  of  aircraft  systems 

-  Monitoring  of  pilot  actions  and  their  compliance  with  those  required  by  the  actual  flight  plan 

In  figure  4,  the  normal  mode  of  operation  is  depicted  which  is  mainly  investigated.  For  this  mode,  which  was  favored  in  the 
first  place,  an  extensive  test  program  is  conducted.  Figure  4  shows  the  potential  pilot  inputs  into  the  cockpit  assistant  and  the 
data  flow  with  regard  to  the  monitoring  function.  The  advices  or  warnings,  as  generated  by  the  MON,  are  tansferred  via  the 
speech  interface.  This  is  one  mode  among  a  number  of  other  possible  ones  which  can  be  distinguished  by  the  degree  of  charg¬ 
ing  the  cockpit  assistant  with  tasks  otherwise  performed  by  the  pilot.  In  case  of  pilot  incapacitation  the  ASPIO  functions  can 
be  used  for  autonomous  emergency  flight. 


3.  IMPLEMENTATION  AND  EVALUATION  EXPERIMENTS 

For  the  experimental  investigation  the  cockpit  assis  tant  is  implemented  in  a  flight  simulator  facility  with  a  one-seat  fixed  base 
cockpit  (fig.5).  Besides  aircraft  dynamics  (6-degrees  of  freedom  model  of  the  HFB  320),  autopilot  (AP),  radio  navigation  sys¬ 
tems  and  wind  characteristics  are  simulated  in  this  facility.  The  pilot  interface  includes  computer-generated  outside  vision, 
artificial  stick  force  as  well  as  speech  input  and  output  in  order  to  closely  match  the  speech  dialogue  in  a  two-man  cockpit 
environment.  As  an  option,  instead  of  pressing  keyboard  buttons,  the  speech  system  can  also  be  used  by  the  pilot  for  all  kinds 
of  inputs  into  the  aircraft  systems.  The  functions  of  the  cockpit  assistant  are  implemented  in  a  VAX  and  a  PS/2  computer. 
Some  implementation  parameters  are  given  in  table  I. 

The  experiments  were  aimed  at  proving  enhancements  in  overall  system  performance,  also  with  regard  to  safety,  pilot  work¬ 
load  balancing  and  pilot  acceptance  under  the  most  possible  realistic  conditions.  For  this  purpose,  the  following  criteria  were 
investigated: 

-  Accuracy  of  flight 

-  Pilot  errors  in  situation  assessment  and  system  operation 

-  Duration  and  quality  of  planning  processes 

-  Pilot  workload 

-  Pilot  acceptance 

The  pertinent  parameters  were  evaluated  for  a  number  of  test  runs  of  IFR-approaches.  These  test  runs  were  organized  in  the 
way  as  shown  in  table  2.  Three  different  IFR  scenarios  [7]  were  developed  which  consisted  of  standard  situations  as  well  as 
of  unusual  events  and  emergency  situations.  Scenario  1  is  exclusively  standard.  Therefore,  it  was  possible  for  this  scenario  that 
all  pilots  flew  both  test  runs,  with  and  without  cockpit  assistant  (CA).  However,  in  scenario  2  and  3  planning  and  decision¬ 
making  is  triggered  by  unforsceable  events.  Each  pilot  could  only  have  one  test  run  in  either  of  these  scenarios,  either  with 
or  without  cockpit  assistant.  A  total  of  nine  professional  pilots,  everyone  with  a  great  amount  of  IFR  flight  experience,  have 
performed  test  runs  for  the  three  different  scenarios.  A  great  amount  of  experimental  data  has  been  provided.  Some  of  the 
evaluation  results  are  presented  in  the  following. 

As  an  indicator  for  flight  accuracy,  the  airspeed  deviation  was  considered  to  be  most  appropriate  for  evaluation.  Figure  6 
shows  typical  time  histories  for  scenario  1.  Figure  6a  illustrates  the  deviations  occurring  without  the  cockpit  assistant,  as  op¬ 
posed  to  figure  6b,  which  shows  much  smaller  deviations  for  the  case  of  activated  cockpit  assistant.  Both  flights  are  performed 
by  the  same  pilot.  The  evaluation  results  for  the  standard  deviation  for  all  pilots,  as  shown  in  figure  7,  demonstrate  a  highly 
significant  improvement  in  flight  accuracy  for  the  flights  with  activated  assistance  functions.  For  scenario  2  the  results  were 
even  better,  for  scenario  3  not  quite  as  good  as  for  scenario  1. 

Pilot  errors  could  be  directly  and  unambiguously  detected.  One  can  say  that,  by  definition,  no  major  deviations  from  normal 
flight  could  be  observed  for  the  flights  with  activated  monitoring  function  of  the  cockpit  assistant.  However,  without  cockpit 
assistant,  pilot  errors  were  detected,  also  more  serious  ones. 

The  time  needed  for  planning  and  decision-making  was  determined  by  the  difference  in  time  between  a  triggering  ATC  in¬ 
struction,  which  also  demanded  for  a  reply  about  the  pilot’s  further  intentions,  and  the  pilot’s  reaction.  In  figure  8,  the  pilot’s 
decision  time  without  support  by  the  cockpit  assistant,  following  an  unexpected  ATC  message  in  scenano  2,  is  shown  in  com¬ 
parison  to  the  time  elapsed  until  pilot  reaction  for  acknowledgement  of  the  flight  plan  recommendation  of  the  cockpit  as¬ 
sistant  was  detected.  The  planning  and  decision  task  was  to  check  for  an  alternative  approach  procedure  under  consideration 
of  weather  and  airport  data,  after  a  message  was  received  about  the  breakdown  of  the  instrument  landing  system  at  the  desti¬ 
nation  airport.  The  differences  are  obvious.  In  addition,  the  pilots  stated  in  the  debriefing  session  that  all  decisions  recom¬ 
mended  by  the  cockpit  assistant,  made  sense. 

The  pilot  workload  was  assessed  by  means  of  subjective  rating  and  the  introduction  of  a  secondary  task.  For  subjective  rating, 
the  SWAT-method  (Subjective  Workload  Assessment  Techmque  [8])  was  used.  As  a  secondary  task,  periodical  tapping  [9] 
was  specified.  The  results  show  a  slight  reduction  of  workload,  but  without  significance. 

Pilot  acceptance  was  evaluated  from  pilot  statements  in  a  questionnaire  which  was  furnished  by  the  pilots  after  their  test 
flights.  Among  other  ratings,  the  technique  of  semantic  differential  [10)  was  used,  as  shown  in  figure  9.  The  median  values 
show  positve  mean  reactions,  i.e.  good  acceptance  by  the  pilots.  The  neutral  overall  assessment  for  the  component  "not  dis¬ 
tracting/  distracting"  was  explained  by  the  pilots  by  certain  lack  of  familianzation  in  system  handling,  in  particular  with  respect 
to  the  specific  speech  recognition  system  used  for  these  experiments. 


4.  CONCLUDING  REMARKS 

A  knowledge-based  cockpit  assistant  for  piloting  tasks  in  IFR  aviation  can  offer  great  benefits  in  monitoring,  planning  and 
decision-making  for  complex  situations.  The  system  can  rapidly  offer  recommendations  to  the  pilot  without  getting  ’tired’  or 
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loosing  information.  The  recommendations  are  derived  by  evaluation  of  alternative  solutions  against  objective  criteria.  The 
presented  system  specially  is  intended  for  the  single-pilot  IFR  operation.  However,  most  of  its  essential  functional  elements 
can  be  applied  for  the  two-man  cockpit  as  well.  The  system  is  able  to  understand  the  situation  on  the  basis  of  knowledge 
about  facts  and  procedures  of  the  piloting  task  environment  and  the  actual  data  about  the  flight  status  and  pilot  actions.  The 
situation  can  be  evaluated  with  regard  to  disturbing  impacts  on  the  flight  plan  and,  if  necessary,  it  derives  a  revised  flight  plan 
as  a  recommendation  to  the  pilot.  It  also  can  serve  the  pilot  for  plan  execution  tasks  in  a  similar  way  as  the  co-pilot  is  working 
in  the  two-man  crew.  Speech  dialogues  as  known  from  the  two-man  cockpit  remain  essentially  unchanged  because  of  the  in¬ 
tegration  of  a  pilot  interface  with  speech  input  and  output. 

The  flight  simulator  experiments  for  testing  the  cockpit  assistant  under  realistic  conditions,  including  the  air  traffic  environ¬ 
ment,  lead  to  the  conclusion  that  this  kind  of  system  actually  can  complement  human  capabilities  and  is  accepted  by  the 
pilots.  Flight  accuracy  was  significantly  improved,  adverse  consequences  of  pilot  errors  were  eliminated  and  also  planning  and 
decision-  making  was  improved.  It  could  be  shown  that  pilot  workload  was  slightly  reduced,  although  the  pilot  interface  was 
not  optimized  at  this  stage  of  development. 
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Figure  1 :  Structure  of  cockpit  assistant 
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Figure  2:  Part  of  syntax  structure  for  speech  recognition 


a:  without  cockpit  assistant 


Figure  6:  Typical  time  history  of  airspeed 
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Cuused  by  specific  features  of  speech  system 


Figure  9:  Questionnaire  result  ror  the  semantic  differential  The  cockpit  assistant  is...’ 
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RESUME 

Cet  article  presents  uri  systems  de  determination  automatique  de  silhouette  d’un 
vehicule  sur  imagerie  infra-rouge. 

Cette  reconnaissance  s’appuie  sur  1 ’uti lisation  explicits  d’un  models  de  I*’ objet  b 
identifier.  Le  formal isme  objet,  mis  en  oeuvre  pour  representer  le  modeie  permet  de 
depasser  le  cadre  d'une  simple  model isation  gbombtrique  et  photombtrique  par 
1 ’ introduction,  des  la  phase  de  conception  du  models  de  connaissances  sur  le 
fonctionnement  du  systems. 

Le  processus  de  reconnaissance  est  fonde  sur  un  schema  general  de  prediction- 
verification  d’hypotheses  et  exploits  un  ensemble  de  primitives  de  type  "regions 
homogenes"  afin  de  rbaliser  1 '  interpretation  de  1‘ image.  Enfin,-  un  mecamsme 
d’ interaction  entre  le  module  d' interpretation  et  le  processus  de  segmentation  bas- 
niveau  permet  de  controler  la  phase  d' am61 loration  de  la  silhouette  en  utilisant  les 
informations  du  modeie. 


INTRODUCTION 


Un  des  axes  pnncipaux  du  dAveloppement  des  performances  des  systemes  d’armes  modernes 
concerne  1 ’amel loration  de  leurs  capacites  sensorial les . 

L’ identification  automatique  ou  semi-automatique  d’objectifs  constitue  l’une  dss 
fonctions  principales  des  systemes  de  veille,  de  reconnaissance,  ou  bien  des 
autodirecteurs  de  missiles. 

Le  systeme  pr6sent6  dens  cet  article  se  place  plus  particulierement  dans  le  cadre  de 
cette  dernibre  application.  Dans  le  domains  des  missiles  autonomes  tirbs  A  distance  do 
sbeuritb,  1 ’ identification  de  l’objectif  devient  une  nbcessitb,  tout  particul ibrement 
dans  le  cas  d'une  cible  mobile  pour  laquelle  un  simple  guidage  inertiel  ne  saurait  6tre 
suff isant. 

La  fonction  d’acquisltion  automatique  de  cible  s’avbre  bgalement  primordiale  afin  de 
mettre  en  oeuvre  des  traitements  de  rbsistance  aux  contre-mesures. 

L’objectif  du  systbme  RUBIS  (RUle-Based  Identification  System)  est  la  determination 
automatique  de  la  silhouette  d’un  vbhicule  sur  imagerie  infra-rouge.  L’extraction  de 
silhouette  s’inscrlt  dans  un  cadre  plus  large  concernant  1 'ensemble  de  la  fonction 
d’acquisition  automatique  d’objectif. 

Le  concept  d'acquisition  d’objectifs  de  petite  taille  (abronefs,  vbhicules  terrestreo, 
ciblos  pour  lesquelles  aucune  preparation  de  mission  n’est  effectube)  recouvre  un 
ensemble  de  fonctions  de  complexitb  croissants  :• 

-  la  detection  ; 

-  la  reconnaissance  ; 

-  1  ’  identification. 


La  detection  de  cibles  consists  en  une  recherche  d’objets  dans  1’ image  pouvant 
constituer  des  objjctifs  potentials.  Les  critbres  de  discrimination  pris  en  compte  lore 
de  cette  btape  sont  relativement  rudimentai res  et  les  algorithroes  de  traitement  des 
images,  pour  la  bande  spectrale  infra-rouge,  sont  gbnbralement  fondbs  sur  une  extraction 
de  points  chauds  et  le  pistage  de  ceux-ci . 


La  reconnaissance  de  cibles  a  pour  objectif  la  classification  des  cibles  potentielles 
sulvant  leur  catbgorie  et  'e  rejet  des  fausses  alarmes  et  des  laurres. 


L’ identification  des  cibles  constitue  la  phase  ultimo  du  processus  d’acquisition  et  a 
pour  finalitb  la  dbtermination,  pour  chaque  objet,  de  son  type. 


La  dbtermination  de  silhouette  consists  A  localiser  prbclsbment  l’objectif  en  extrayant 
parfaitement  sa  forme  dans  1’image.  Cette  opbration  est  effectube  sane  utiliser  de 
connaissances  a  priori  sur  Vbchelle  et  sur  1 ’orientation  de  1 ’objet  dans  1’ image. 


1 
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RUBIS  met  en  oeuvre  dee  techniques  issues  du  domains  de  1* intel 1 igence  Artificielle 
afin  de  rAaliser  une  segmentation  initials  da  IMmage  en  regions  puis  d’effectuer 
1 '  interpretation  de  cet  ensemble  de  primitives  en  utilisant  un  module  du  vAhicule  A 
identifier. 

Le  processus  d’ interpretation  suit  un  sch6mu  general  de  prediction-verification 
d’hypotheses  en  utilisant  les  elements  remarquables  de  l’objet  pour  6mettre  une 
hypothese  de  vision  (point  de  vue,  ecnelle,  position  de  i’objet  dans  1’ image).  La  phase 
de  verification  tente  ensuite  de  confirmer  bu  d’infirmer  cette  hypothese  par 
identification  precise  de  cheque  element  constituent  Is  v6hicule, 

L'affinage  progressif  de  1 ’ interpretation  peut  conduire  a  la  definition  de  requAtes  de 
recherche  particul lAres  de  regions  dans  le  cas  d’objets  manquants  ou  mal  aegmentbs  lors 
de  la  partition  initials.  Cette  Interaction  du  module  d‘ interpretation  avec  les 
traitements  de  segmentation  permet  d’amAliorer  sensiblement  les  performances  de  RUBIS. 

Nous  prAsenterone  successivement  le  module  de  segmentation,  le  processus 
d’ interpretation  ainsi  qua  le  schema  de  definition  des  moUAles  puip  none  donnerons  das 
rpsultats  quantitatifs  obtenus  sur  une  base  d’une  trentaine  d1 images  FLIR  0-12  ^im 
rspresentant  un  blinde  sous  un  ensemble  d’attitudes  variAes. 


LA  SEGMENTATION  EN  REGIONS 


La  tAche  de  ce  module  consists  a  fourmr  au  processus  d’ interpretation  un  ensemble  de 
primitives  suff isamment.  precises  pour  entreprendre  la  mise  en  correepondsnce  ontro  les 
indices  visuals  extraits  de  1’ image  et  les  objets  ou  parties  d’objets  du  models  de 
reference. 


Le  choix  des  primitives  de  type  region  est  justifie  par  1' appl ication.  II  s’agit  en 
effet  d* identifier  des  v6hicules  relativement  mal  rBsolus  dans  1 ’ image  ;  lee  dimensions 
de  leur  silhouette  n’excedont  pas  quelques  dizaines  de  pixels.  L’uti 1 isation  d’un  mode 
d’ intsrpretation  fondd  sur  des  primitives  gAometriques  (segments  do  droite)  ne  saurait 
convenir  car  la  precision  des  primitives  est  trop  faible.  Pour  un  segment  de  longueur 
egale  a  10  pixels,  une  erreur  de  posi tionnement  de  1  pixel  sur  chaquo  extrAmite  conduit 
A  une  erreur  de  11"  sur  1 ’orientation  du  segment.  Par  eilleurs,  la  repartition  dec 
gradients  d’intensite  aur  la  surface  de  la  cible  ne  rend  pas  toujours  corooto  des 
diffArentes  parties  du  vAhicule  et  les  segments  cie  contour  obtenus  n’ont  pas 
d’homologues  dans  un  models  de  type  CAO. 


La  definition  des  cibios  en  termes  d’assemblagea  de  regions  de  luminance  homogene 
permet  d’atteindre  la  robustesae  indispensable  A  notre  application.  Toutefols, 
1 ’uti  1  leation  d’une  representation  en  regions  de  l’lmago  ne  fourmt  pas  suffisament 
d’ informations  pour  etre  en  closure  de  calculer  pr6cis6ment  1 ’orientation  de  la  cible. 
Oans  le  caa  des  objectifs  mobiles  do  dimensions  rAduites,  cette  information  n’est  pas 
nAceasaire  car  la  ccnnaissance  approximative  de  l’attitud9  de  l’objet  dona  l’image  est 
8uffisants  pour  local iser  le  point  d’ impact  optimal. 

Le  module  de  segmentation  est  divise  en  deux  ensembles  principaux.  D’une  part,  un 
traitoment  de  segmentation  er.  regions  fonde  sur  une  approche  de  division  recursive  et 
d’autre  cart  une  base  de  regies  de  production  qui  analysent  localement  1 ’ensemble  des 
regions  et  dAclenchent  1 ’appl  ication  des  actions  correctness  de  division  ou  de  fusion. 


Las  traitements  de  segmentation  initials  sont  fondAs  sur  deux  princlpes  importants  : 

-  un  seuillage  multiple  des  niveaux  de  gris 

-  une  approche  recursive  des  divisions. 


Ce  processus  est  initialise  par  la  recherche  d'un  seui 1  sur  1’imag*  initiate.  Le  seui 1 
est  applique  sur  1 1  image  et  une  premiere  partition  an  regions  connexes  est  determines. 
Pour  chacune  des  regions  ne  satisfalsant  pas  le  critAre  d’uniformite,  1 ’ensemble  des 
operations  de  division  est  r6it6r6  selon  un  processus  rAcursif, 
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La  determination  du  seuil  pour  une  region  sst  fondd  sur  la  mdthode  de  KOHLER  [1]  qui 
exploits  1 'histogramme  des  transitions  moyennes  de  la  'dgion.  Cat  histogramme  est 
determine  de  la  faqon  suivante  :• 

Pour  tout  couple  (p,g)  de  pixels  adjacents  tels  qua  L(p)  (  L(q) 

si  L(p)  <  n  <  L( q )  N(n)  =  N(n)  +  1 

C(n)  =  C(n)  +  Min(L(q)  -  n,  n  -  L(p)) 


H(n)  =  C(n)  /  N(n) 

ou  L( )  est  la  fonction  luminance. 

La  valeur  de  seuil  S  correspond  au  niveau  de  gris  pour  lequel  H(S)  atteint  son  maximum. 

Le  seuil  s61ectionn6  correspond  A  la  maximisation  du  nombre  de  transitions  moyennes 
qu’il  met  en  evidence. 

L’approche  recursive  permet  de  determiner  et  d’appliquer  les  seui Is  localement  et  de 
mimmiser  ainsi  les  effets  de  propagation  des  segmentations  obter.ues  par  application 
de  seui Is  globaux. 

L’analyse  des  segmentations  obtenues  montre  que  des  defauts  subsistent.  Ces 
imperfections  peuvent  Stre  classdes  en  trois  categories  principles  : 

-  fragmentation  excessive  des  regions  :  une  zone  ayant  une  existence  sdmantique 
(representation  d’un  objet  physique)  est  divisde  an  de  nombreuses  regions  qui  n’ont 
aucune  real  ltd  physique  ; 

-  phenomena  d’ "overmerging"  :  certaines  regions  rocouvrent  partial  lament  un  objet  ainsi 
qu’une  partie  de  1 'envi ronnament  ; 

-  erreur  de  placement  de  la  frontiers  d’une  region  :  ceci  est  essential  lament  do  au 
profil  du  gradient  au  voisinage  de  la  frontiers  des  zones  homogdnes.  Ce  profil  n’est  pas 
parfait,  et  c6la  occasionne  des  erreurs  sur  1 ’estimation  de  la  position  du  maximum  du 
gradient. 


Ces  ddfauts  ont  deux  causes  essential les  : 

-  le  enters  de  segmentation  est  mal  adaptd  a  la  zone  traitee  (ceci  est 
particulidrement  vr a  sur  les  images  prdsentant  a  la  fois  des  zones  homogenes  et  des 
zones  texturdee)  ; 

-  1 'analyse  de  1 ' image  est  trop  globale  et  ceci  conduit  a  la  determination  de 
pararndtres  de  segmentation  localement  mal  adaptes. 

Pour  pallier  ces  inconvdments,  nous  avons  choisi  d’amdliorer  la  segmentation  initials 
par  des  traitements  contrdlds  par  des  bases  de  regies.  Les  premisses  de  ces  regies  de 
production  ont  pour  but  la  detection  d’une  configuration  mteressante  afin  de  ddclencher 
dans  la  partie  action  un  traltement  adaptd. 

La  mise  en  oeuvre  d’un  dispositif  de  contrdle  de  ces  traitements  est  ndeessaire  afin 
d’dviter  que  les  actions  de  division  et  de  fusion  ne  se  suceddent  inddfiniment  sur  une 
portion  de  1 ’image.  Ce  systdrae  de  segmentation  a  en  effet  un  comportement  non-monotone 
dont  il  est  impossible  d’assurer  la  termlnaison  a  priori.  Nous  ddtai  Herons  done  la 
methods  de  contrdle  que  nous  avons  mise  en  place  et  qui  est  basde  sur  la  repartition  de 
pararndtres  d’dtat  sur  cheque  objet  manipuld  par  le  systdme. 

La  representation  dee  primitives  de  Is  segmentation  est  rdalisde  dans  un  formalisms 
objet.  Deux  classes  principales  d’objets  sont  utilisdes  par  le  systdme  de  segmentation 
bas-mvaau  :• 

-  la  classe  REGION 

-  la  classe  RELATION  D’ADJACENCE 

Cheque  rdgion  est  reprdsentde  par  une  lists  d’attributs  et  par  son  extension  iconique 
(liste  des  pixels  qu’elle  contient).  Nous  distinguons  trois  types  d’attributs  principaux 
:  108  attribute  photomdtriques  (lids  A  la  rdpartitlon  des  mveaux  de  gris  sur  la  surface 
de  la  rdgion),  les  attributs  gdomdtrlques  et  les  attribute  morphologiques  (lids  A  la 
forme  de  la  rdgion). 

Outre  les  informations  propres  A  cheque  rAgion,  nous  sommes  amends  ti  utiliser  au  cours 
du  ddroulement  du  processus  de  segmentation  des  informations  sur  les  liaisons  entre  les 
rdgions  lorsqu’il  s’agit  de  ddterminer  les  couples  de  rdglons  adjacentee  candidates  A 
une  fusion,  nous  avons  done  choisi  de  reprdsenter  les  relations  d’adjacence  entre  deux 
rdgions  par  un  objet  du  systdme  qui  possdde  des  attributs  sur  la  frontidre  entre  les 
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deux  regions  (longueur  du  p6rim6tre  commun,  moyenne,  valeurs  extremes  et  6cart-type  du 
gradient  le  long  de  la  frontiers). 

Les  regies  de  segmentation  sont  orgamsSes  en  paquets  correspondant  au  type  d’action 
qu’elles  appllquent.  Le  premier  paquet  regroups  les  regies  de  fusion.  Au  nombre  de 
qulnze,  ces  regies  effectuent  des  fusions  selon  des  oriteres  photometriques  ou 
morphologlques. 

Les  oriteres  photometriques  font  intervemr  prlncipalement  ,  la  luminance  moyenne  de 
cheque  region,  une  information  de  texture  (mesurSe  a  l’aide  des  attribute  de  repartition 
du  gradient  calcuies  sur  l’mt6rieur  de  la  region  pour  eviter  les  defauts  116s  6  la 
proximite  de  la  frontiers  de  la  region). 

Les  oriteres  geometriques  et  morphologiques  aont  fond6s  essentiel lement  eur  l’aire  des 
r6gionc  (tout  particul ierement  sur  les  rapports  d’aires  entre  deux  regions  adjacentes 
candidates  6  la  fusion),  et  sur  des  attribute  li6s  A  la  forme  des  regions  qul  permettent 
de  detector  dee  configurations  particul lbres  comma  ,  par  example,  l'existenoe  de  regions 
particul ierement  fines  (par  le  calcul  des  moments  d’inertie,  du  ratio  entre  l’aire  de 
la  region  et  l’aire  de  son  erode  etc...).  La  regie  suivante  est  un  example  de  cette 
utilisation  des  attribute  : 

Deux  regions  fines  adjacentes 
dont  1’une  est  peu  etendue 

dont  les  axes  principaux  d’inertie  sont  paral Idles 
dont  la  longueur  de  pbrimatre  commun  est  faible 

sont  fusionn6es 


Deux  modes  principaux  de  division  sont  mis  en  oeuvre  dans  RUBIS.  Le  premier  mode 
consists  A  activer  selectivement  sur  une  seule  region  la  segmentation  par  seuillage 
fondAe  sur  la  methode  de  KOHLER.  Cette  methods  de  segmentation  est  activ6e  par  les 
conditions  suivantea 

-  grande  dynamique  de  mveaux  de  gris  ; 

-  1 'existence  de  plusieurs  modes  sur  1  ’ histogramme  en  mveaux  de  gris  de  la  region  •; 

-  la  presence  de  gradients  assez  Sieves  dans  cette  region. 

Le  second  mode  de  division  est  appe!6  “Division  Morphologique".  Cette  action  consists 
6  dlviser  une  region  selon  un  masque  obtenu  par  ouverture  morphologique.  Cette 
operation  permet  d'isoler  les  zones  fines  ou  les  l rr6gularit6s  de  forme  apparaissant  sur 
la  region  et  d’effectuer  1 'operation  de  division  en  Isolant  ces  zones.  Ce  traitement  a 
pour  but  de  supprimer  les  zones  de  r6tr6cissement  d’une  region.  Ce  phenomena  est 
g6n6ralement  occasionnA  par  une  discontinuite  du  gradient  tr6s  locale  (souvent  limitee 
a  1  ou  2  pixels)  le  long  de  la  frontiers  d’une  region.  La  simplification  de  la  forme  des 
regions  constltue  un  gage  de  reussite  pour  le  processus  d’ interpretation  qui  aura  a 
effectuer  une  mise  en  correspondance  entre  des  regions  issues  de  1’ image  et  les  objets 
du  models  qui  sont  souvent  ddcomposables  en  formes  relativement  simples.  Cette  division 
est  activbe  sur  des  regions  de  compacitd  (aire  I  p6rim6tre’'  )  faible. 


Le  contrdle  du  processus  de  segmentation  est  guide  par  les  quelques  principes  suivants 


-  Maltriser  les  ph6nom6nes  d’  "ovorspl itting"  et  d’ “overmerging”  ; 

-  Assurer  la  terminaison  du  processus  de  segmentation  ; 

-  “Pr6venir  plutdt  que  Gu6rir“  :  II  s'agit,  d’une  part,  d’Aviter  des  actions 
catastrophiques  qui  remettraient  en  cause  le  fonctionnement  du  systems  et  d’autre  part 
de  preparer  la  mise  en  correspondance  en  isolant  dds  la  phase  de  creation  des  regions 
des  objets  qui  paraissent  caractdristlques. 

Le  contrdle  du  processus  de  segmentation  peut  6tre  envisage  a  deux  mveaux.  L’approche 
globale  consists  a  effectuer  une  evaluation  sur  1 ’ensemble  des  regions  afin  de  calculer 
un  enters  de  satisfaction  vie  a  vis  du  traitement  de  segmentation.  Cette  evaluation 
globale  a  pour  but  d’orianter  la  stategie  de  segmentation  a  haut-mveau  qui  consists  a 
prlviiegler  un  type  particul ier  d’actions  sur  une  zone  de  T image.  Cette  stratbgie  de 
haut-n'Weau  dolt,  etre  accompagn6e  d'un  mode  d’ appl ication.  Ceci  revient  a  determiner 
pr6cis6ment  le  e6quencement  des  d6clenchements  des  regies  en  associant  a  chacune  d’elles 
la  ou  les  primitives  de  V  image  a  modifier.  Cette  fo»-me  de  contrdle  a  Ate  mise  en  oeuvre 
par  Nazif  et  Levine  [2],  [3],  [4],  [53,  [6]  qui  ont  dSveloppe  un  algorithms  de  contrdle 
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fonde  sur  la  thAorie  des  ensembles  flous. 

La  mise  en  oeuvra  d’un  systAme  da  contrAle  global  conduit  A  da  nombreux  calcula  afin 
da  remettre  A  jour  r6gul iArement  la  critAre  d’apprAciation  da  la  segmentation.  Las 
principales  oaractAristiques  prises  en  oompte  lors  da  catte  evaluation  sont  las 
suivantes  : 

-  1 ’uniformitA  da  chaque  region  ; 

-  la  contraste  inter-rAgion6. 

L’approche  adoptde  dans  la  systAme  RUBIS  consiste  A  dAlocaliser  la  fonction  da  controls 
sur  chaque  objet  manipulA  par  la  systAme  A  base  da  rAgles.  Chacune  des  regions  da  la 
segmentation  comporte  dans  sa  lists  d’attributs  un  attribut  d’etat  qui  permet  da  noter 
las  actions  admissiblas  sur  catte  region.  Alnsi  cat  attribut  peut  prendre  las  valeurs 
suivantes. 

-  libra  :  pour  indiquer  qu'il  n’existe  aucune  restriction 

-  indivisible  :  catte  valeur  signifie  qua  la  region  ne  peut  Atre  divisea.  11  exists 
deux  variantas  pour  catte  caracteristiqua  qui  concernent  la  methods  da  division  (une 
region  peut  6tre  indivisible  par  seuillage  ou  par  division  morphologique  ou  totalement 
indivisible)  ; 

-  galea  :•  aucune  action  n’est  admise  sur  la  region.  Ceci  signifie  qua  la  region  est 
indivisible  at  qu'elle  ne  peut  Atre  fusionnAe  avec  aucune  da  ses  regions  adjacentes. 

Par  a^lleurs,  nous  avons  la  possibilite  d’intardire  une  fusion  particuliAre  pour  un 
couple  da  deux  regions  adjacentes  en  notant  cette  caracteristiqua  sur  un  attribut  d’etat 
da  1 ’objet  reprAsentant  laur  relation  d’adjacence. 

Cette  organisation  du  controls  permet  d’implanter  aussi  bien  una  strategic  globale  qua 
locale.  Dans  RUBIS,  nous  nous  sommes  llmites  A  1 ’ implantation  d’une  stratAgie  locale 
monotone.  En  effet,  1 'affectation  d’une  valour  A  1 ’attribut  d’etat  d’une  region  n’est 
pas  remise  en  cause  lors  du  processus  de  segmentation  en  regions  excepts  pour  remplacer 
sa  valeur  courante  par  une  valeur  plus  restrictive.  Ainsi  une  region  dont  l’etat  est 
"morpho-indivisible"  peut  passer  A  l’etat  “indivisible"  male  en  aucun  cas  A  l’etat 
"libre".  Cette  utilisation  des  attributs  d’etat,  associAe  A  la  diminution  des 
intervalles  de  tolerance  des  critAres  de  segmentation  assure  la  convergence  du  processus 
de  vision  bas-niveau. 

Cette  approche  ne  conduit  pas  A  une  partition  de  1  ’  image  optimale  vis  A  vis  des 
critAres  devaluation  SnoncSs  par  Nazif  et  Levine  mais  il  s'agit  1A  d’un  compromis  entre 
un  rSsultat  acceptable  et  une  charge  de  calcul  excessive. 

Par  aineurs,  1 'ensemble  des  regions  obtenu  eet  destine  A  Atre  interprets  par  le 
processus  de  vision  haut-niveau  qui  utilise  un  modAle  explicits  de  1 ’objet  A 
reconnattre.  Au  cours  de  cette  interpretation,  la  partition  en  regions  mitiale  ne  sera 
plus  simp  lament  AvaluSe  par  rapport  A  un  ensemble  de  critAres  independents  du  contenu 
de  1’ image  mais  vis  A  vis  d’un  modAle  de  1 ’objet.  Celui-ci  sera  utilise  pour  Amettre  des 
hypotheses  de  resegmentation  locale  afin  de  rechercher  des  objets  manquants  ou  de 
prScIser  las  limites  des  objets  identifies  afin  d’amAliorer  les  performances 
d’ identification. 

II  est  extrAmement  ambitieux  et  mAme  lllusoire  de  chercher  A  conetruire  un  traitement 
g6nSral  de  segmentation  independant  de  la  final  its  de  Sexploitation  qui  sera  faite  de 
1 ’ensemble  des  primitives  extraites  de  1 ’image. 

Dans  le  cas  de  RUBIS,  il  n’est  pas  primordial  de  traitor  parfaitement  les  zones  A  trAs 
forte  texture  car  ellee  reprSsentent  gAn6ralement  des  zones  de  1 ’environnement  natural 
sur  lesquelles  aucun  traitement  d’ interpretation  n’est  effectuA. 

L'unlque  traitement  des  regions  texturAes  consiste  A  detector  1 ’existence  oe  la  texture 
par  une  mesure  de  repartition  des  gradients.  La  region  est  isolAe  en  uti ’leant 
1 'attribut  d’etat,  afin  d’Avitor  qu’une  action  de  division  ne  soit  effectuAe  car  cela 
conduirait  A  un  nombre  trAs  important  de  regions  et  A  la  saturation  totals  du  systAme. 
Ceci  constitue  la  meilleure  illustration  du  troisiAme  principe  "PrAvenlr  plutAt  que 
guArir". 

DAs  la  phase  de  segmentation,  des  informations  alnsi  que  la  connaissance  de  la  finalite 
du  systAme  sont  nAcessaires  afin  d’optimiser  les  performances  du  module  de  vision  bas- 
niveau.  Le  schema  de  controls  utilise  dans  RUBIS  a  permis  d’implanter  via  un  ensemble 
de  mAta-rAgles  des  critAres  de  gel  de  certaines  regions  prAsentant  des  oaractAristiques 
dlecrlminantes  vis  A  vis  de  1'objectif  de  RUBIS,  A  savoir  “ 1 ’ identification  de 
vi;. ' cules" .  Les  regions  de  luminance  et  de  compacitA  AlevAes  sont  IsolAes  mAme  si  leur 
aire  en  fait  des  candidates  A  la  fusion.  Ces  regions  reprAsentant  des  elements  chauds 
du  vAhlcule  ou  de  1 'environnement  (leurres,  feux,  etc...)  et  constituent  des  germes  pour 
le  systAme  d’ interpretation  qui  initialise  le  schema  de  prediction-verification 
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d’hypothAses  sur  ce  type  d’objets  remarquables, 

L’attrlbut  d'etat  est  Agalement  utilise  comma  compte-rendu  des  actions  de  division 
d’une  region.  Si  un  mods  de  division  ne  donne  aucun  resultat  sur  une  region,  cette 
information  est  conserves  via  1’attribut  d'etat,  ce  qui  permet  d’6v1ter  une  invocation 
de  cette  action  sur  cette  meme  region  alors  que  1 'issue  est  vouee  A  l’echec. 

(.'ensemble  de  primitives  obtenu  A  1’ issue  de  la  phase  de  segmentation  n’est  en  aucun 
cas  optlmale  ("Un  objet,  une  region")  et  les  traitements  de  haut-mveau  doivent  Stre 
capables  de  rAaliser  la  reconnaissance  malgre  les  dAfauts.  Le  choix  des  seuils  et  de  la 
strategle  pour  le  processus  de  bas-niveau  ne  saurait  eiiminer  un  type  de  defaut  :  en 
effsi,  la  suppression  totals  du  ph6nomAne  d’overmerging  conduirait  A  un  nombre 
prohibitif  de  regions  tout  particu I lArement  sur  des  zones  A  forte  texture  mais,  par 
ailleirs,  1 'overmerging  est  extrSmement  prAjudici able  au  processus  d’ interpretation  qui 
dbbute  1 ’ identification  du  vAhicule  A  1’aide  des  informations  dAduites  de  quelques 
regions  supposbes  representatives  d'un  objet  du  modAle. 

Ceci  montre  la  nAcessaire  collaboration  entre  les  diffArents  niveaux  d’un  systems  de 
vision  artificial le  lorsque  le  domains  d' appl icatlon  est  relativement  vaste  (scenes 
naturelles,  conditions  de  prises  de  vue  variables,  envi ronnements  complexes,  masquages, 
contre-mesures  etc. . . ) . 


LA  MODELISATION  DES  CI6LES 

L‘ identification  des  cibles  est  rAalisAe,  dans  le  systems  RUBIS,  par  un  module  de  mlse 
en  correspondance  entre  un  modAle  du  vAhicule,  dont  le  type  est  suppose  connu,  et 
1 'ensemble  de  rAgione  mises  en  evidence  lore  de  la  pnase  de  segmentation  bas-niveau.. 

La  rAussite  de  cette  Atape  de  reconnaissance  est  extrAmement  dependants  de  la  qualitA 
du  modAle  fourm.  L’evaluation  de  cette  qualite  peut  etre  realises  vis  A  vis  des 
cri tAree  suivants  : 

-  la  precision  et  la  finesse  de  la  description  ; 

-  la  robustesse  de  la  modAlisation  qui  rend  compte  de  son  adaptabilite  A  des  conditions 
d’ observations  variAee  (point  de  vue,  qualite  des  images)  ; 

-  la  facilitA  de  mise  en  oeuvre  du  modAle  par  le  processus  d’ interpretation. 

Nous  avons  done  cherchA  A  dAfinir,  dans  RUBIS,  un  cadre  de  modAlisation  peu 
contraignant,  robuste,  et  capable  de  prendre  en  compte  1 'aspect  tridimensionnel . 

L’approche  retenue  est  un  modAle  multi-bidimensionnel .  Cela  sigmfie  que  la 
modAlisation  d’un  vAhicule  est  composAe  d’un  certain  nombre  de  modAles  616mentaires  de 
vues  bidimenslonnel  les  du  vAhicule  selon  plusieurs  orientations  caractAristiques.  Compte 
tenu  de  la  resolution  des  Images  et  du  champ  couvert  par  la  cible  sur  1’ image  ,  nous 
nous  sommes  1  unites  A  cinq  orientations  caractAristiques  correspondent  A  une  vue  de 
face,  une  vue  de  profil,  une  vue  d'arriAre,  une  vue  de  trois-quart  face  et  une  vue  de 
trois-quart  arriAre. 

Chacun  de  ces  modAles  AlAmentaires  est  decompose  en  un  certain  nombre  d’objets  qui  sont 
les  representations  d’une  partie  du  v«hicule  selon  1 'orientation  du  modAle  AlAmentaire 
considArA.  Ainsi,  le  modAle  AlAmentaire  d'un  blinde  de  type  AMX30  selon  une  vue  arriAre 
sera  constituA  de  six  objets  : 

-  deux  pots  d’Achappement  droit  et  gauche  ; 

-  deux  chenilles  droite  et  gauche  : 

-  la  vue  arriAre  du  chassis  ; 

-  la  vue  arriAre  de  la  tourelle. 

Le  modAle  du  vAhicule  est  reprAsentA  dans  un  formalisms  objet  en  utilisant  les  notions 
d’hAritage  afin  de  construire  une  arborescence  pour  chacun  des  elements  constituent 
cette  cible.  Cette  structure  arborescente  regroups  1 ’ensemble  des  vues  d’unmAme  AlAment 
du  vAhicule  selon  1 ’ensemble  des  orientations  dAcrltes  dans  le  modAle. 

Cette  hierarehlsatlon  de  la  description  d’un  objet  consists  A  regrouper  au  niveau  le 
plus  AlevA  de  gAnAralite  l’ensemble  des  caractAristiques  communes  A  toutes  les 
representations  de  1 ’objet  dans  les  niveaux  rnfArieurs.  Par  exemple,  quelque  soit 
1 ’orientation  du  vAhicule  sur  1’image,  le  pot  d’Achappement,  s’ll  est  visible,  sera 
reprAsentA  par  une  ou  plusieurs  rAgions  de  luminance  AlevAe  ;  cette  caractSristique 
commune  sera  conservAe  au  niveau  le  plus  AlevA  de  1 ’arborescence.  Cette  hiArarchie  est 
dAcrite  en  termes  de  classe  et  de  sous-classsc  afm  de  tirer  pleinement  parti  de  la 
notion  d’hAritage  inhArente  A  1a  representation  orientAe  objet. 
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II  exlate  deux  types  de  critfcres  pour  caract4riser  un  objet  : 

-  Lea  crit4res  dlts  "  intnns6ques"  car  leur  verification  ns  fait  intervemr  que  des 
attribute  propres  aux  regions  concernees  par  cat  objet.  II  s’agit  par  example  de  la 
luminance,  de  la  compaclte,  de  1 ’elongation. 

-  lea  critbres  appeies  "relations  de  voisinage"  qui  decrivent  les  conditions  que 
1 'objet  doit  verifier  vis  4  vis  des  objets  voieins. 

La  model isation  des  vehicules  comporte  done  une  double  structuration  ;  d'une  part  une 
arborescence  sur  chacun  des  elements  et  d'aut’re  part  des  relations  de  voisinage  entre 
les  representations  de  ces  elements  [7]. 

Le  souci  de  robustesse  qui  a  guide  la  t&che  de  modeiisation  nous  a  conduit  4  introduire 
dans  la  modeiisation,  d’une  part  la  notion  de  granularite  variable,  et  d’autre  part  4 
definir  des  la  conception  du  models  des  hypotheses  de  gestion  des  echecs  pouvant 
survenir  lors  de  la  mise  en  correspondance. 

La  notion  de  granularite  variable  est  introduite  afin  de  rendre  compte  d6s  la  phase 
d’etabl issement  du  models  des  phenomenes  de  pertes  de  resolution  suivant  la  distance  de 
la  cible  au  capteur.  La  perte  de  resolution  a  deux  causes  principales  :• 

-  le  nombre  de  pixels  apparaissant  4  la  surface  de  la  silhouette  de  la  cible  qui  est 
une  consequence  directs  de  1 ’eloignement  de  l’objet  ; 

-  la  fonction  de  transfert  de  l’optique  qui  conduit  4  une  diffusion  apparente  de  la 
cible  dans  1’ image,  4  une  perte  de  nettete  des  details  et  4  des  phenomenes  de  fusion 
entre  les  zones  proches  de  luminosite  peu  differente. 

D’apres  ces  observations,  ll  apparatt  clairement  que  le  models  d’un  vehicule  situe  4 
longue  distance  ne  peut  pas  etre  deduit  du  modeie  d’une  vue  4  courts  distance  par  simple 
reduction  d’echelle,  L ' introduction  de  plusieurs  modeies  correspondant  4  une  meme 
orientation  du  vehicule  permet  de  prendre  en  compte  ces  diff6rents  aspects  tout  en 
definissant,  d6s  la  conception  du  models,  des  hypotheses  de  gestion  des  echecs  du 
processus  d' interpretation.  Un  example  de  cette  organisation  est  fourni  par  la 
modeiisation  du  char  de  type  AMX30  "vue  arriSre’’.  Sur  une  image  de  grande  resolution, 
on  peut  distinguer  les  regions  correspondant  aux  deux  pots  d' Schappement  ainsi  qu’une 
region  ldgdrement  morns  lumineuse  reprdsentant  le  chassis.  Dans  le  ca6  du  m6me  vehicule 
vu  selon  la  meme  orientation  4  une  6chelle  beaucoup  plus  faible,  ll  est  rarement 
possible  de  distinguer  ces  trois  regions  qui  n’en  forment  plus  qu’une  seule.  La  prise 
en  compte  de  ce  type  de  phenomena  lors  de  la  phase  de  construction  du  models  fournit  au 
processus  d’ interpretation  plusieurs  alternatives  de  raisonnement.  Ceci  6vite  d’avoir 
4  envisager  des  mises  en  correspondance  entre  une  fusion  de  plusieurs  objets  du  modeie 
et  d'une  seule  region  de  1’ image  ou  bien  de  chercher  4  diviser  une  region  de  1’ image 
sans  information  a  priori  sur  le  mode  de  division  4  mettre  en  oeuvre. 


L’ INTERPRETATION 

Le  module  d’ interpretation  est  fonde  sur  un  schema  general  de  mise  en  correspondance 
entre  les  objets  du  modeie  et  1 'ensemble  des  primitives  extraites  de  1’ image.  La  mise 
en  correspondance  est  effectuee  par  un  processus  de  prediction-verification 
d'hypotheses.  Cheque  identification  d’un  element  du  models  donne  lieu  4  la  creation  d’un 
objet  instance  de  la  classe  reprSsentant  1 'element  reconnu.  Cette  instance  stocks  la  ou 
les  primitives  correspondant  4  1 'element.  La  construction  d’ instances  du  modeie  au  fur 
et  4  mesure  de  la  reconnaissance  permet  de  g4rer  plusieurs  interpretations  de  fagon 
parallels. 

Dans  le  cae  le  plus  general  d’une  mise  en  correspondance  entre  un  models 
tridlmensionnel  et  une  image  bldimensionnel le,  l’hypothese  de  vision  est  un  sextuplet 
de  valeurs  (a,e,d,X, Y,r)  correspondant  respecti vement  aux  deux  angles  d’azimuth  et 
d’eievation  de  la  ligne  de  visde  dans  le  repere  cible,  4  la  distance  capteur-but,  4 
la  position  du  centre  de  la  silhouette  et  4  1 'orientation  de  la  silhouette  dans  1’ image. 

Dans  notre  application,  nous  avons  n4glig6  l’angle  t  en  faisant  1’hypothSse  que  le 
capteur  est  stabilise  en  roulis  et  qie  le  sol  est  suffisament  horizontal  pour  supposer 
t  =  0.  Par  ailleurs,  nous  n'avons  mc741is6  qu’un  ensemble  restreint  d’orientatlons  de 
la  ligne  de  visbs  par  rapport  4  la  c  ble  :■  ■* 

-  vue  arrifcre  (a  =  -  90’,  e  =  C  i 

-  vue  de  face  (a  =  O’,  e  =  O' ) 

-  vue  de  profll  (a  =  90’,  e  =  O’ ) 

-  vue  de  trois-quart  arrISre  (a  =  -  45",  e  =  0’ ) 

-  vue  de  trois-quart  face  (a  =  45',  e  =  O') 


Nous  n’avons  jamais  tenu  compte  dans  notre  modeiisation  de  1 ’angle  e.  Les  modules 
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construits  pour  les  cinq  orientations  priviiegides  peuvent  etre  utilises  sans  aucune 
modification  pour  des  angles  e  <  20 

Le  parametre  d  de  1’hypothese  de  vision  est  remplacd  dans  notre  systems  par  l’6cheile 
E  du  vehicule  sur  1’ image. 

La  prediction  d’hypotheses  consiste  a  utiliser  certainas  primitives  de  l’image  et  a 
tenter  de  les  mettre  en  correspondance  avec  des  objets  du  modaie.  Da  chacune  de  ces 
tentatives,  sont  ddduits  les  paramatres  d’une  hypoth6se  de  vision  a  savoir  une  attitude 
de  1 'objet,  une  position  approximative  de  sa  silhouette  et  une  premiere  estimation  de 
1 ’dchel le. 

Ce  processus  de  prediction  ost  fonde  sur  des  objets  particuliers  du  modaie  appaies 
"Hots  de  confiance  du  modaie"  (I.C.M).  Ces  objets  seront  utilises  comma  des  germes  sQrs 
afin  d’emettre  des  hypotheses  de  vision.  Ces  objets  doivent  poss6der  les  proprietSs 
suivantes  : 

-  ils  sont  identif iables  sans  connaissance  a  priori  su  I’echelle  et  1 ’orientation  de 
la  cible  dans  1 ’ image  ; 

-  ils  sont  discriminants  afin  d’orienter  le  choix  du  modaie  bidimensionnel  a  utiliser 


-  ils  presentent,  si  possible,  des  caracteristiques  suffisamment  distinctes  par  rapport 
aux  objets  voisins  afin  d’etre  isoies  par  le  module  de  segmentation  en  regions.  Ceci 
permet  d’obtenir  comme  representation  de  cet  objet  dans  1’ image  une  seule  region 
facilement  identifiable  a  partir  de  laquelle  11  est  possible  d’obtenir  une  premiere 
estimation  de  l’achelle  du  vehicule  dans  l’image. 

La  representation  des  ICM  dans  le  modaie  fait  appel  a  la  notion  d’heritages  multiples. 
Une  classe  ICM  est  erase  et  est  1’une  des  classes  antecedentes  des  classes 
representatives  des  objets  qui  jouent  le  rble  d’ICM. 

Les  objets  a  forte  emissivite  thermique  constituent,  dans  le  cas  de  notre  application, 
d’excellents  ICM.  Alnsi,  pour  un  blinde  de  type  AMX30,  les  pots  d ’ echappement  et  les 
chenilles  (partlcul iarement  vues  de  face  ou  d’arriare)  constituent  les  principaux  ICM 
de  la  model isation. 

Le  deroulement  du  m6canisme  d’ interpretation  debute  par  une  recherche  de  toutes  les 
regions  de  1’image  verifiant  1’un  des  critares  de  selection  des  ICM  ;  ces  crltares  etant 
regroupea  au  niveau  de  la  classe  ICM.  Pour  chacune  de6  primitives  ainsi  identifi6es, 
sera  tentee  une  mise  qn  correspondance  avec  1’une  des  classes  “filles"  de  la  classe  ICM 
et  ainsi  de  proche  en  procho  jusqu’a  une  classe  "feuille"  de  1 ’arborescence. 

Chaque  mise  en  correspondance  est  6valu6e  par  un  calcul  de  qualit.6  qui  tient  compte 
d’une  part  de  1 ’adequation  entre  les  attnbuts  propres  des  primitives  et  les 
caracteristiques  intrlnsSques  de  l’objet  du  modaie  et  d’autre  part  de  la  presence  ou  de 
1 ’absence  des  elements  voisins  dSc-its  dans  le  modaie.  La  propagation  descendante  des 
mi  see  en  correspondance  est  accompagnee  d’une  remise  4  jour  de  l’indice  de  qualite  en 
tenant  compte  de  la  valeur  trouv6e  au  niveau  precedent  et  des  nouveaux  critares  verifies 
au  niveau  considers. 

Lorsqu’une  hypothSae  est  emise,  le  processus  de  mise  en  correspondance  est  active  pour 
chacun  des  objets  appartenant  au  modaie  cholsi.  Une  phase  de  prediction  permet  de 
limiter  la  zone  du  recherche  de  cet  element  grace  aux  informations  d6duites  de 
1 ’ identif 1  cation  de  1’ICM. 

L’archltectu-e  de  modaie  permet  de  maintenir,  paral laiement,  plusieurs  lignes  de 
raisonnement  au  cours  d’une  interpretation.  II  est  parfois  impossible  de  conclure  entre 
deux  interpretations  (par  exemple  :•  vue  de  face  ou  vue  de  trois-quart  face  d’un  memo 
vdhicule)  mais  le  systbrne  indique  tout  de  mama  les  elements  correctement  reconnus  m*me 
si  leurs  caracteristiques  ne  sont  pas  exactement  semblables  4  cel les  correspondant  4 
une  alternative  unique. 

Lorsqus  plusieurs  ICM  d’un  meme  vehicule  sont  simultanement  visibles  sur  une  image, 
plusieurs  interpretations  cohdrentes  sont  obtenues.  Dans  ce  cas,  un  mecanlsme  de 
propagation  de  contraintes  est  mis  en  oeuvre.  Cette  operation  est  realises  en  appliquant 
les  contraintes  de  volsinage  issues  de  1 ’ identif ication  d’un  ICM  sur  1 ’ interpretation 
real  1  see  en  utilisant  le  second  ICM  comme  germs. 

Le  stade  ultime  du  processus  d’ interpretation  concerns  1  interaction  entre  le  module 
de  vision  haut-niveau  et  le  systime  de  segmentation  bas-nlveau  [8],  Cette  Interaction 
poursuit  deux  buts  *  d’ur.s  part  affiner  «t  ordciser  les  limltes  de  la  silhouette  et 
d’autre  part  tralter  les  cas  d’dchecs  (absence  apparente  d’un  objet). 

Cette  amelioration  consiste  4  appliquer  localeient  des  operations  de  division  en 
choisl8eant  la  zone  d’effet  grace  4  1 ’ interpretation  obtenue  par  le  proceeeue  de  haut- 
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niveau. 

L’ensemble  des  objets  correctement  identifies  dans  1’ image  eet  utilise  afin  de  calculer 
la  position  precise  d’un  certain  nombre  d’axes  caractAristiques  du  models  AlAmentaire 
utilise.  A  ce  models  est  associe  un  prototype  constltud  de  l’ensemble  des  positions 
standard  des  axes  pour  une  echells  de  reference.  La  comblnaison  des  axes  de  reference 
et  des  axes  determines  dans  1’ image  conduit  au  calcul  d’une  fenetre  de  recherche  pour 
chaque  objet  du  modeie  616mentai re.  La  position  et  la  tallle  de  cheque  fenetre  sent 
extremement  precises  car  ce  calcul  prend  en  compte  simultan6ment  toutes  les  informations 
Issues  de  1 ’ Identification  de  plusleurs  objets. 

Chaque  fenetre  est  utilises  pour  determiner  la  liste  des  regions  susceptibles  de 
representer  I’objet  concerne.  Un  traltement  de  division  est  applique  aux  regions  ayant 
une  Intersection  avec  la  fendtre  mais  Agalement  une  partie  de  la  surface  situAe  hors  de 
la  fenetre.  Pour  chacun  des  fragments  mis  en  evidence  par  1 ’operation  de  division,  le 
systems  cherche  A  les  associer  A  un  objet  du  models.  L’opdration  est  reitArAe  jusqu’A 
1  ’  identification  ds  1 'objet  cherche  ou  bien  jusqu’au  constat  d’un  echec  qul  sigmfie 
3oit  que  1 'objet  est  masque  soit  que  1 'aspect  sous  lequel  ll  apparait  n’est  pas 
model isA. 

L’uti 1 isation  du  models  pour  Amettre  des  requAtes  de  segmentation  bas-niveau  permet 
de  minimiser  les  Imperfections  inevitables  ds  la  partition  initials  en  regions  et 
d’amAliorer  de  maniAre  significative  les  performances  du  systems. 

L'Avaluation  quantitative  des  rAsultats  a  Ate  realises  sur  une  base  d’une  trentaine 
d’ images  reprAsentant  un  blindA  sous  un  ensdmble  d'attitudes  variAes.  Cette  evaluation 
a  Ate  menAe  en  comparant  la  silhouette  expArimentale  determines  par  RUBIS  avec  la 
silhouette  thAorique  extraite  interactivement  par  un  opArateur. 

Deux  crltAres  d ’ aoprAci ation  sont  pris  en  compte.  Le  coefficient  Cl  qui  represents  le 
pourcentage  de  surface  de  la  silhouette  thAorique  correctement  inclus  dans  la  silhouette 
expArimentale.  Le  second  coefficient  Cd  mesure  le  pourcentage  de  surface  de  la 
silhouette  expArimentale  n’ appartenant  pas  A  la  silhouette  thAorique. 

Les  rAsultats  ont  Ate  AvaluAs  avant  et  aprAs  la  phase  de  resegmentation.  Le  coefficient 
Ci  Agal  A  71*  avant  resegmentation  passe  A  78*  aprAs  cette  Atape.  Le  coefficient  Cd 
passe  de  7*  A  5*.  Sur  la  plupart  des  images  traitAes,  le  mAcamsme  de  resegmentation 
accroit  la  performances  dans  des  proportions  trAs  faibles  de  l’ordre  de  1*  mais  dans 
certains  cas  difflciles  d’environnement  structure,  cet  affinage  amAliore  sensiblement 
la  precision  de  la  silhouette  (Jusqu’A  20  *  d’ augmentation  de  la  valeur  de  Cd).  Ceci 
peut  avoir  des  consequences  importantes  sur  la  precision  de  localisation  du  point 
d’ impact  optimal . 

Le  systems  RUBIS  a  AtA  dAveloppA  sur  une  station  de  travail  de  type  SUN  3/260  en 
langages  C  et  Common  Lisp.  L’ensemble  des  rAglSB  pilotant  le  module  de  segmentation  en 
regions  a  AtA  implantA  A  1’aide  du  gAnArateur  de  systAme  expert  KIRK  et  le  langage 
objst  utilise  pour  l’ensemble  du  projet  est  le  produit  FLAME.  Ces  deux  outils  fondAs  sur 
Common  Lisp  sont  deux  produits  dAveloppAs  par  TH0MS0N-CSF. 


I 
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CONCLUSION 

La  maquette  actual le  de  RUBIS  a  permis  d'une  part  de  prouver  la  faisabilitA  d’un 
systAme  autonome  de  reconnaissance  de  cibles.  Elle  a  constituAe  un  banc  d’essai  pour 
un  certain  nombre  de  mAthodes  et  d'outils  issue  du  domains  de  1 ’ Intel  1 lgence 
artificlelle  qul  fournissent  au  concepteur  un  envl ronnement  de  dAveloppement  souple  et 
Avolutif  afin  de  tester  et  de  prototype  rapidement  les  algorithmes. 

Les  rAsultats  obtenus  sur  la  base  d’lmages  de  test  sont  probants  car  RUBIS  rAalise 
dans  la  plupart  des  cas  une  determination  de  silhouette  corrects  malgre  la  presence  de 
fonds  complexes.  Ces  performances  ont  deux  raisons  essentlelles  :  1'approche  multi- 
crltAres  de  la  segmentation  en  regions  et  la  tolerance  du  module  d’ interpretation  vis 
A  vis  des  defauts  de  segmentation  grAce  A  la  collaboration  des  deux  niveaux  du  systAme 
par  1 'Amission  de  requAtes  de  segmentation  A  parti r  du  processus  de  mise  en 
correspondance. 

RUBIS  a  en  outre  un  potential  d' evolution  important  car  11  const. tue  un  cadre  general 
pour  supporter  des  dAveloppements  ultArleurs  tant  au  niveau  de  la  segmentation 
(utilisation  du  mouvement  par  example)  qu'au  niveau  de  1 ’ interpretation  (strategies  de 
recherche,  extension  vers  un  nombre  plus  importants  de  types  de  vAhicules). 

La  prise  en  compte  de  1 ’aspect  temporal  semble  conttituer  l’axe  de  dAveloppement  le 
plus  prometteur.  Ceci  peut  se  tradulre  dans  le  processus  de  segmentation  par 
1 'uti 1 isation  du  mouvement  (segmentation  du  champ  des  vitesses)  ou  en  propageant  les 
rAsultats  obtenus  pour  1’ image  N  sur  1’ image  N+l  afin  d’affiner  ainsi  la  segmentation 
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au  lieu  de  reinitial iser  1e  processus  pour  chaque  nouvelle  image.  Oarts  le  processus 
d’ Interpretation,  la  mise  en  oeuvre  d’un  models  devolution  temporal  le  de  1 'aspect  de 
la  cible  dans  1’ image  devrait  permettre  un  eiagage  considerable  de  l’arbre  des  solutions 
pour  les  mises  en  correspondance  et  augmenter  aigmf icativement  les  performances  tant 
du  point  de  vue  de  la  precision  que  d’un  point  de  vue  efficacite,  point  fondamental  dans 
la  perspective  d'une  application  embarquee. 
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Sunnary 


In  the  future,  the  cooperation  of  pilot  and  controller  will  change.  Technical 
advances  contributing  to  this  change  are  a  more  intelligent  airborne  Flight  Management 
System  (FMS)  and  a  datalink  connecting  the  FMS  and  Air  Traffic  Control  (ATC).  Against 
this  background,  concepts  for  an  Experimental  Flight  Management  System  and  its  human- 
centered  system  design  approach  are  described  Combined  with  a  basic  scenario  of  future 
Air  Traffic  Management  (ATM)  the  requirements  for  an  airborne  constraint  management 
subsystem  are  developed.  The  fundamentals  of  aircraft  route  planning  and  system 
operation  including  considerations  on  the  interaction  between  constraint  management  and 
man-machine-interface  are  discussed  and  an  on-line  algorithm  for  aircraft  route 
planning  is  presented.  The  paper  also  describes  the  present  state  of  a  software 
prototype  and  the  soft-  and  hardware  employed. 


1 .  Introduction 


Flight  Management  Systems  have  become  an  essential  part  of  civil  aircraft  operation 
in  the  last  decade.  Increasing  research  efforts  on  automation  regarding  future  Air 
Traffic  Management  (ATM)  reveal  opportunities  to  improve  the  support  of  the  air  traffic 
controllers  by  utilizing  airborne  resources.  Future  FMSs  and  ATM  systems  will  have  a 
datalink  module  and  thus  enable  direct  data  communication  between  airborne  and  ground 
equipment.  Undoubtedly,  such  a  datalink  combined  with  planning  and  decision  support  for 
the  air  traffic  controller  will  increase  system  performance.  Flight  precision  as  well 
as  exploitation  of  airspace  will  increase  considerably.  Combined  with  increasing 
'intelligence'  of  the  supporting  system  the  human  operator  demands  additional 
explanation  on  how  machine-generated  proposals  are  generated  and  what  the  implications 
are  on  system  operation  and  aircraft  safety.  However,  the  changing  role  of  the  human 
operator  in  this  environment  probably  introduces  new  errors  (Wiener  [1]). 

In  the  past  numerous  papers  have  been  published  outlining  fundamentals  of  a  future 
'intelligent  cockpit'.  References  [2]  to  [53  present  typical  examples.  The  paper  by 
Rouse,  Geddes  and  Curry  [2]  provides  a  detailed  summary  regarding  functional  design  and 
architecture  of  the  man-machine-interface  m  case  of  a  complex  system.  It  is  chosen  as 
a  basic  reference.  In  this  context  detailed  formatting  aspects  of  information  to  be 
displayed  as  well  as  selection  of  appropriate  operator  input  devices  are  of  minor 
interest.  However,  the  design  of  an  operator  support  system  requires  a  wholistic  view 
to  create  an  efficient  overall  system.  System  architecture,  algorithm  design,  data 
structures  and  appropriate  selection  of  task-related  information  in  total  can  provide 
high  performance  and  acceptance  (Hacker  [6]).  Simply  spoken,  the  questions  -  what  is 
needed,,  how  can  it  be  achieved  and  which  realisation  features  fit  human  capabilities  - 
depend  on  each  other  and  have  to  be  answered  nearly  simultaneously  to  pro , ids  a 
profound  design  approach. 

The  paper  concentrates  on  this  comprehensive  aspect  of  system  design.  The  aircraft 
route  planning  task  of  the  pilot  as  well  as  the  constraint  management  subsystem  are 
discussed  making  direct  reference  to  the  man-machine- interface .  An  ATM  scenario  and  a 
FMS  conceptual  design  proposal  pointing  to  future  needs  are  presented  for  the 
definition  of  functional  requirements  as  well  as  for  the  implementation  of  a  software 
prototype. 


2.  Flight  Management  Concept 


Prerequisite  for  the  implementation  of  an  aircraft  route  planning  algorithm  is  the 
FMS.  Therefore,  the  conceptual  model  of  an  experimental  FMS  currently  designed  as  well 
as  a  scenario  for  future  integrated  ATM  are  described. 


cimental  Flight  Management  System 


The  following  description  refers  to  the  Conceptional  Model  of  an  Experimental 
Flight  Management  System  defined  by  the  working  group  'Experimental  Flight  Management 
System  { EFMS ) 1  within  the  "Programme  for  Harmonized  Air  Traffic  Management  Research  m 
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EUROCONTROL  (PHARE)"  [7,8].  The  participating  European  Research  Institutions  -  BFS,, 
CAA,  CENA,  DLR,  NLR ,  RAE ,  RSRE  and  EUROCONTROL  -  specified  an  experimental  system, 
which  aims  at  high  flexibility  to  serve  as  a  research  tool  for  datalink,  air  traffic 
management,  profile  prediction  and  optimisation  as  well  as  man-machine-interface 
studies  and  experiments.  A  major  goal  of  the  programme  will  be  the  realisation  of  an 
ATM  demonstrator  including  real  aircraft  operation  in  an  advanced  experimental 
environment . 


This  system  proposal  may  serve  both  as  an  example  for  an  advanced  conceptual  design 
of  a  future  FMS  and  as  basis  for  an  experimental  implementation 


The  system  consists  of  the  eight  modules  shown  in  figure  1  performing  the  following 
functions : 


-  Man-Machine-Interface 

Translates  objects  of  the  EFMS 
Network  into  Man-Readable 
Messages  and  vice  versa 

-  Constraint  Manager 

Maintain  Active  and  Secondary 
Constraint  List  (i.e.  an 
extended  issue  of  the 
Flight  Plan) 

Respond  to  Constraint  Edits 

Respond  to  Activate  Secondary 
Constraint  List  Command 

-  Profile  Prediction  and  Outer 

Loop  Guidance 

Construct  the  Active  and 

Secondary  Flight  Profile 

Generate  and  Broadcast  the 

Outer  Loop  Guidance  Vector 

-  Data  Base  Manager 

Extract  Data  from  the  Data 

Base  following  to  a  query 
and  respond  to  the 
initiating  element 

Update  Data  Base 


Figure  1  -  EFMS  Functional  Elements 


-  Data  Link  Manager 

Gateway  between  the  Air-Ground  Datalink  Network  and  the  EFMS  Network 

The  remainder  provides  the  connection  between  the  EFMS  network  and  aircraft 
specific  environment: 

-  Automatic  Flight  Control  System  (AFCS)  Interface 

Generate  Implementation  Specific  Guidance  Commands  for  Output  to  AFCS 


-  Navigation  System 

Interface  between  specific  Navigation  Equipment  and  EFMS  Network 
Selection  and  Management  of  Navigation  Sensors 
Autotuning  and  Data  Conditioning 


-  Sensor  Manager 

Interface  between  Aircraft  specific  Sensors  and  EFMS  Network 

Data  Conditioning,  Filtering,  Interpolation 

Before  proceeding  to  the  details  of  system  operation  two  definitions  are  formulated 
[8]: 

A  constraint  consists  of  a  waypoint  combined  with  attributes  specifying  position, 
altitude,  speed,  time  and  a  constraint  type.  Optional  attributes  such  as  maximum  speed 
or  altitude  limitations  may  be  added  if  required.  The  resultant  constraint  list  may  be 
characterised  as  an  extended  issue  of  a  flight  plan. 

The  profile  or  tra lectorv  is  computed  from  the  constraint  list,  meteorological  data 
and  aircraft  performance  data.  Combined  with  lateral  and  vertical  deviation  limits  a 
'tube'  is  obtained.  Time  constraints  result  m  the  conception  of  a  'bubble'  moving 
through  the  'tube'.  'Bubble'  and  'tube'  represent  the  contract  between  aircraft  and 
ATC.  The  'bubble'  denotes  the  manoeuvre  space  of  the  aircraft  in  which  the  aircraft  is 
authorised  to  optimise  its  own  trajectory.  The  trajectory  tnen  is  the  basis  for 
computing  the  outer  loop  Guidance  vector,  which  is  fed  into  the  AFCS. 

Strategical  planning  operation 

The  operation  sequence  for  strategical  route  planning  is  initiated  by  the  pilot 
with  the  input  of  waypoints,  thus  creating  a  flight  plan  which  is  the  basis  of  the 
constraint  list  completed  by  default  values  and  any  limitations  from  the  database.  The 
flight  profile  generation  module  computes  from  this  data,  meteorological  forecast  data, 
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ATC  constraints  received  via  datalink,  aircraft  performance  data  and  navigation  data  an 
optimized  trajectory.  The  trajectory  is  displayed  to  the  pilot.  If  it  is  appropriate 
the  pilot  may  request  a  clearance  for  that  trajectory.  ATC  will  return  the  take-off 
time  and  -  if  necessary  -  a  number  of  additional  constraints.  If  the  ATC  constraints  do 
not  conflict  the  airborne  constraint  list,  the  pilot  may  activate  the  trajectory.  If 
not,  the  pilot  has  to  adhere  to  the  ATC  constraints  and  to  initiate  another  run  of  the 
flight  profile  generator  to  get  an  appropriate  trajectory.  The  clearance  request  is 
repeated.  The  described  operation  is  iterative  and  will  be  repeated  until  an  agreement 
between  pilot  and  controller  is  achieved.  Then,  ATC  will  return  the  trajectory  and 
deviation  limits,,  which  define  the  4-D  'tube'  m  space. 

Alternatively  the  pilot  may  wish  to  alter  constraints.  This  initiates  an  identical 
negotiation  process  between  the  pilot  and  the  controller,,  but  will  include  the 
essential  ATC  constraints  of  the  current  flight  plan. 

Tactical  operation 

Tactical  control  commands  such  as  'holding',  'step  climb',  'descent  to'  or  'direct 
to'  require  immediate  action  by  the  pilot  Then  the  pilot  has  to  enter  modifications  of 
the  constraint  list,  start  the  profile  generation  process  and  subsequently  to  initiate 
the  negotiation  process  with  ATC. 

At  present,,  different  definitions  of  the  time  span  referring  to  tactical  control 
are  discussed.  Here,  a  time  span  of  about  30  minutes  ahead  is  assumed.  Short  term 
tactical  control  m  case  of  emergency  conditions  will  be  handled  most  probably  via  R/T- 
communication. 


2.2  Air  Traffic  Management  Scenario 


The  future  ATM  scenario  proposed  by  a  GARTEUR  working  group  outlines  a  concept  of 
operation  [9]  which  is  generally  characterized  by  computer  assistance  of  the 
controller,  a  datalink  between  aircraft  and  ground  equipment  as  well  as  suitably 
equipped  aircraft,  i.e.  FMS  and  even  4-D  navigation  capability. 

Three  distinct  levels  of  datalink  communication  between  aircraft  and  ATC  are  of 
importance  in  this  respect: 

The  background  level  is  characterised  by  an  automatic  exchange  of  information 
without  human  intervention.  This  level  includes  general  aircraft  data,  such  as  aircraft 
call  sign,  aircraft  type,  equipment  identification  etc.,  and  initialises  the  datalink 
communication  process.  Subsequently,  exchange  of  dynamic  data  at  fixed  time  intervals 
predominates  datalink  communication.  Regarding  the  downlink,  position  as  well  as 
meteorological  data  with  reference  to  time  and  aircraft  position  will  be  included.  The 
uplink  communication  may  contain  broadcast  weather  data,,  ATC  constraints  and  runway 
data. 

The  strategical  level  represents  pilot  or 
controller  initiated  exchange  of  long-term 
related  strategic  planning  information. 

The  tactical  level  requires  an  immediate 
responso  by  pilot  or  controller. 

Both  strategical  and  tactical  level  refer 
to  the  trajectory  negotiation  process  where  the 
tactical  level  assumes  the  higher  priority. 


Figure  2  illustrates  the  data  types  and 
datalink  communication  [9].  Corresponding  to 
these  basic  items  ATC  is  in  a  position  to 

-  perform  collective  optimisation  to 
resolve  conflicts  between  individual 
preferred  profiles,  where  clearances  take 
the  form  of  a  ngourously  defined  'tube' 
in  space, 

-  exploit  improved  navigation  m  case  of 
suitably  equipped  aircraft  by  direct  use 
of  aircraft  position  data  or  because  the 
aircraft  can  comply  with  clearances 
unaided, 


-  maintain  a  database  providing  current 
meteorological  data  in  a  three 
dimensional  grid  within  the  geographical 
region  of  interest. 

With  suitably  equipped  aircraft  as  well  as  ground-based  optimisation  procedures  air 
traffic  density  could  be  raised  to  maximum  capacity  without  reducing  safety  demands. 
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Figure  2  *  Date  Typas  and  Datalink  Corrrr.'jnicatSon 
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Poorly  equipped  aircraft  limit  the  maximum  capacity  because  they  have  not  the 
capability  to  operate  in  »  comparable  and  narrow  'tube'. 


3.  Functional  Requirements  for  Constraint  Management 


Constraint  Management  represents  a  central  function  of  the  FMS.  Therefore, 
functional  requirements  for  constraint  management  relate  to  the  operation  of  the  entire 
FMS,  regarding  internal  constraint  handling  and  including  man-machine-interface 
aspects.  A  wholistic  approach  is  needed. 


3.1  System  Operation 


The  constraint  management  module  gathers  the  constraint  inputs  of  the  pilot  and 
creates  the  constraint  list  which  is  the  basis  for  subsequent  data  processing  within 
the  profile  prediction  module.  The  result  is  displayed  to  the  pilot  and  -  in  case  of 
pilot's  agreement  -  Wj.ll  be  sent  to  ATC  via  the  datalmk.  Further  constraints  added  by 
ATC  will  restart  the  process  of  editing  the  constraint  list  by  the  pilot.  After 
activation  of  the  constraint  list  and  the  inherent  trajectory  computation,  the  pilot 
supervises  aircraft  performance  and  the  progress  of  flight  with  regard  to  aircraft 
state,  navigation  state  and  outer  loop  guidance.  A  constraint  list  modification  may 
result  in  a  considerable  increase  of  workload,  since  supervision  of  the  currently 
active  and  modified  constraint  list  has  to  be  done  in  parallel. 

Even  such  a  simplified  description  of  the  pilot  task  irrespective  of  a  possible 
work-sharing  between  the  two  crew  members  asks  for  some  assistance  for  the  pilot. 

Assistance  for  route  planning  can  be  achieved  by  means  of  airlme-def ined  routes 
created  off-line  and  stored  m  the  database  or  by  implementing  a  route  planning 
algorithm  for  on-line  use.  In  the  first  case  each  airline  emoloys  a  set  of  tailored, 
airline-specific  routes.  The  second  case  is  just  a  suggestion  for  the  further 
development  of  the  FMS,,  but  both  solutions  must  be  consistent. 

A  current  restriction  for  system  operation  is  the  limited  number  of  constraint 
lists  handled  by  the  FMS  To  solve  this  problem  future  systems  must  contain  several 
constraint  list  buffers.  On  the  one  hand  this  requires  a  complete  set  of  buffer 
manipulation  commands  such  as  "COPY",  "DELETE"  and  others.  On  the  other  hand  priority 
values  for  the  operational  sequence  have  to  be  introduced  to  the  system:  A  tactical  ATC 
message  as  well  as  the  subsequent  modification  of  the  active  constraint  list  demand  a 
higher  priority  than  the  route  planning  process  of  the  next  flight  leg,  which,  in  turn, 
affects  the  piofile  prediction  process. 

The  most  frequently  addressed  FMS  module 
will  be  the  database  management  unit.  The 
performance  characteristics  of  this  module  will 
influence  the  total  system  performance 
considerably.  The  database  contains  navigation 
data,  aircraft  performance  data  and 
meteorological  data.  The  source  of  the 
navigation  database  which  is  provided  according 
to  ARINo  standards  [10].  An  off-line,  ground- 
based  procedure  will  be  needed  for 
transformation  of  the  navigation  database  into 
a  form  suitable  for  efficient  database  access. 

These  features  increase  the  system 
complexity  considerably.  Efficient  system 
handling  b,  the  pilot,  however,  demands  a  low 
complexity  of  the  system  model  within  the  man- 
machine-mterface.  Therefore,  future  systems 
will  include  operator  models  and  aiding  units 
providing  task  queuing  and  method  selection 
dependent  on  pilot  workload  assessment  [2]. 


3.2  Internal  Constraint  Handling 


The  essential  function  of  the  constraint 
management  module  is  represented  by  the 

'constraint  editor'.  Modification  of  the 

constraint  list  requ.  as  a  se v.  of  euitoi 
commands  optionally  supplemented  by  arguments, 
i.e.  attributes  and  values  for  a  detailed 
definition  of  the  particular  constraint.  Figure  3  outlines  the  command  se"  as  well  as 
necessary  argument  specifications. 

There  are  only  few  basic  commands  needed  ("INSERT",  "DELETE"  and  "CHANGE")  which 
are  to  be  combined  with  a  navigation  obje  t  identifier  as  well  as  optional  attributes 


NAVIGATION 

OBJECT 

IOEMTV1ER 


ATTWBUTE<S)  /  VALUE 


dtpa-lgra,  dattlnatton, 
batfn  of  WflMLplii, 
•no  ol  llohtjj&n. 


DELETE 

CHANOE 


ALTITUDE 

SPEEO 

CLIMB 

DESCENT 

TIME 


rivakJ, 
waypoint, 
pdofdaMrtad- 
wa /point 


O  airway,  SIO,  STAR,  Approach  Rout*,  navald,  vwypolnt, 
prfM^aflftad-waypoW 

Flflurt  3  'Constraint  List  Editor  Commands 
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and  values.  In  case  of  different  attribute  modifications  rather  long  command  sequences 
may  occur.  From  an  ergonomic  point  of  view  it  seems  convenient  to  extend  the  command 
set  by  including  attribute  modification  commands,  e.g.  "te  attributes  ’via1, 
'altitude1,  'speed'  etc.  combined  with  an  identifier  and  a  value.  Because  of  their 
operational  importance,  two  tactical  commands  "DIRECT-TO"  and  "HOLDING"  are  included, 
although  they  actually  may  be  considered  as  attribute  representations  of  the  basic 
commands  "DELETE"  and  "INSERT". 

Other  candidates  for  a  command  set  extension  are  'LATERAL-OFFSET'  and  'AVOID'. 
However,  they  are  not  included  because  they  represent  future  research  topics  to  achieve 
an  efficient  utilisation  of  computer  resources  of  ground-based  and  airborne  equipment. 

Besides  the  constraint  editor  function  a  buffer  manipulation  command  set  has  to  be 
realised.  It  includes  commands  like  COPY,  KILL  or  DELETE,  CONCATENATE;  CHANGE  BUFFER, 
LIST  BUFFER  and  ACTIVATE,,  all  of  them  to  be  combined  with  one  or  two  buffer 
identifiers . 

The  complete  command  set  supported  by  the  constraint  management  module  represents  a 
hierarchy.  Buffer  manipulation  commands  are  comparable  to  operating  system  commands  of 
a  computer,  and  the  constraint  list  editor  is  directly  related  to  text  editing. 
Different  command  input  formats,  if  identified  as  unambiguous,  shall  be  acceptable  for 
the  constraint  management  module,  to  facilitate  the  development  of  different  internal 
models  of  the  command  hierarchy.  Even  the  combination  of  different  commands  shall  be 
allowed. 

A  general,  self-evident  requirement  regards  constraint  identification.  To  end  up 
with  a  consistent  constraint  list  the  constraint  management  software  must  be  able  to 
discriminate  between  different  kinds  of  constraints  m  cooperation  with  the  database 
management  module  A  rather  simple  example  may  be  the  differentiation  between  airways 
and  waypoints  enroute  and  m  the  terminal  area,  pilot-defined  waypoints  as  well  as 
airline  specific  routes  or  even  system-generated  routes  and  waypoints  have  to  be 
classified  appropriately. 

A  formal  error-checking  is  performed  by  the  man-machine-interface  module,  e.g.  a 
check  whether  the  command  is  complete.  An  entry  without  a  navigation  object  identifier 
or  an  attribute  change  without  a  value  make  no  sense.  Tho  constraint  management  module 
then  will  perform  error-checking  with  regard  to  the  entry  in  cooperation  with  the 
database  management,,  e.g.  whether  the  object  identifier  is  available  in  the  navigation 
database  or  whether  attribute  modifications  are  allowed.  Airway  direction,  altitude  or 
airspace  restrictions  may  lead  to  a  rejection  of  an  attribute  modification  by  the 
system. 

To  sort  the  constraint  list  entries,  the  constraint  management  module  needs  the 
information  of  starting  and  endpoint  or  departure  and  destination,  if  the  planned  route 
is  a  complete  flight  leg.  By  means  of  the  route  planning  algorithm  the  constraint  list 
can  be  constructed  without  inserting  any  waypoint  in-between.  The  input  sequence  of 
waypoints  -  even  if  the  pilot  inserts  additional  waypoints  to  guide  the  algorithm  -  is 
of  no  importance. 


Line  SeiPCl  K«y! 


CDU  screen  area 
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With  the  constraint  list  completed  the 
constraint  management  module  may  initiate  the 
profile  prediction  process.  This  process  may  be 
started  automatically  even  if  the  pilot  continues 
entering  additional  waypoints.  The  system  has  to 
verify  whether  such  additional  inputs  are  already 
contained  m  the  constraint  list  r c  -  not  - 
whether  they  r*..  resent  an  increase  of  costs  and 
therefore  lead  to  a  rejection  of  the  additional 
waypoint.  Waypoint  entries  by  means  of  the  "VIA" 
command  must  not  be  refused  by  the  system,  but 
represent  essential  points  of  the  plannee’  route. 


3.3  Man-Machine-Interaction 
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The  I/O-device  for  current  FMSs  is  a  Control 
Display  Unit  (CDU).  which  has  been  standardized  by 
ARINC  Specification  739  [11].  Figure  4  shews  this 
CDU  referring  to  the  EFHS  project  [12).  This  man- 
machine-interface  system  is  characterized  by  a 
display  of  14  lines  of  alpha-numeric  text  including 
title  line  and  scratch  pad,  6  line  select  keys  on 
each  tide  of  the  display,  it  function  keys  and  a 
set  of  alpha-numeric  input  keys.  Mtp/chart 
information  is  directed  ,o  the  navigation  display 
(ND)  of  the  .Electronic  Flight  Instrument  Syscem 
(EFIS).  Modes  of  operation  allow  the  selection  of 
different  information  contents  and  output  formats 
navigation  display.  The  EFIS  control  panel  includes  the  selector  for  several 
modes  (rose,  arc,  plan),  the  range  selector  and  five  optional  data  display 
.  For  each  of  these  selectors  only  one  function  can  be  activated  at  a  time. 


Figure  4  •  Control  Display  Unit  (ARINC  739) 
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However,  alphanumeric  input  requires  a  lot  of  time  and,  unfortunately,  often 
results  in  errors,  even  m  uncritical  situations  [1],  An  advanced  control  display  unit 
will  have  to  include  hardware  features  such  as  graphic  display  and  pointing  device  for 
an  easy  combination  of  graphical  objects  with  desired  functions  [13,14]. 

For  flight  plan  generation  and  aircraft  route  planning,  respectively,  graphic 
objects  denote  map/chart  information  requiring  a  rather  high  resolution  and  respectable 
screen  size.  The  requirements  for  a  vertical  situation  display  are  even  less,  but  the 
vertical  planning  supervision  demands  a  comparable  pilot  'playground',  i.e.  a  graphic 
display  design  and  an  adaquate  pointing  device.  Whether  this  pointing  device  is 
optimally  represented  by  a  rollball  or  a  touch  screen  can  not  be  answered 
satisfactorily  yet. 

A  page-oriented  display  layout  as  realized  in  the  current  display  design  of  the  CDU 
may  be  useful  because  of  the  diversity  of  information  contents.  Probably,  the  applied 
information  structure  of  the  current  CDU  refers  directly  to  the  pilots'  cognitive 
structure  [15].  .A  graphic  display  can  realise  such  a  structure  by  means  of  an 
information  overlay  technique,  comparable  with  slides  containing  an  identical  basic 
information  pattern,  but  different  detailed  information  contents  layed  over  each  other. 
Nevertheless  an  alpha-numeric  display  of  the  constraint  list,  initialisation  data  and 
aircraft  performance  will  be  necessary.  On  the  one  hand  it  facilitates  'a  change  of  the 
third  figure  behind  the  decimal-point',  on  the  other  hand  it  increases  pilots' 
acceptance. 

Another  function  made  available  by  the  control  display  unit  is  the  access  to  the 
map  information  access  for  the  displayed  region  or  range.  The  planning  pilot  will  need 
this  information  about  navigation  objects  not  contained  in  the  constraint  list,  but 
located  close  to  the  planned  track.  Besides  that,  the  pilot  must  be  able  to  change  the 
range  or  reference  coordinates.  These  requirements  have  to  be  supported  by  the  database 
management  module. 

These  considerations  refer  to  a  separate  control  display  unit  including  a  graphic 
display,  pointing  device  and  keyboard.  The  navigation  display  of  the  EF1S  may  remain 
unchanged  m  principle  with  just  minor  modifications.  The  plan  mode  may  be  removed  and 
operation  may  be  simplified  by  discarding  the  optional  data  selector,  if  'essential 
information'  can  be  brought  up  automatically.  In  general,  the  navigation  display  can  be 
used  to  visualize  the  active  constraint  list  as  well  as  tactical  commands  from  ATC. 


4.  Aircraft  Route  Planning 


Search  strategies  applied  on  route  planning  represent  a  classic  item  of  Artificial 
Intelligence  (AI)  exemplified  by  the  famous  'travelling  salesman'  problem,  the  solution 
of  which  identifies  the  shortest  route  between  several  cities  permitting  him  to  visit 
all  cities,  but  only  once  (Nilsson  [16]).  Other  applications  of  AI  search  strategies 
regard  robot  movement  m  dynamic  environment  [17]  as  well  as  aircraft  route  planning 
for  military  missions  [18,19], 

The  problem  domain  of  civil  aircraft  routing  is  slightly  different  from  these 
applications.  Intermediate  waypoints,  airway  segments  and  terminal  area  procedures  have 
to  be  developed  first,  even  the  number  of  intermediate  waypoints  is  unknown,  when 
starting  the  search.  Nevertheless,  search  strategies  are  applied  to  all  these  planning 
problems,  only  optimisation  criteria  and  problem  space  description  will  differ. 

Route  Planning  Algorithm 

The  task  'Aircraft  Route  Planning'  is  performed  m  a  two-phase  cycle.  It  is 
initiated  as  soon  as  the  system  has  knowledge  of  the  start/begm  (or  departure)  and  the 
end  (or  destination)  of  the  pre-planned  flight.  Normally  a  waypoint,  navaid  or  airport 
is  required  for  the  definition  of  ' start/begin' .  The  starting  point  may  be  represented 
by  the  current  aircraft  position  or  a  nominated  route  waypoint. 

Within  the  first  phase  distance  and  track  of  the  direct  route  between  'begin'  ar.d 
'end'  are  calculated.  If  there  are  more  than  these  two  points  available,  e.g.  m  case 
of  a  'via  airway'  command  entering  all  airway  defining  waypoints,  an  identical 
calculation  is  performed  for  each  waypoint  in  relation  to  the  start-point.  Then,  the 
waypoints  are  sorted  to  distance.  With  a  criterion  based  on  distance  and  track 
calculation,  waypoints  being  far-off  are  deleted.  Additionally,  waypoints  occurring 
twice  in  the  list,  for  instance,  in  case  of  airway  intersections,,  are  identified  and 
reduced  to  a  single  list  entry. 

The  first  phase  is  characterized  as  mainly  passive.  Some  basic  calculations 
combined  with  database  queries  are  performed  to  collect  as  much  information  as  possible 
about  the  list  entries.  The  pilot  is  not  informed  about  the  result*  in  particular. 

The  second  phase  is  dominated  by  activo  search.  The  waypoint  list,  created  within 
phase  one  and  probably  incomplete,  represents  the  initial  state  of  search.  Beginning 
with  the  first  list  entry  (starting  point  or  departure)  a  number  of  adjacent  waypoints 
on  the  actual  and  each  intersecting  airway  is  used  for  the  creation  of  a  search  tree. 
Additionally,  those  waypoint  entries  of  the  initial  waypoint  list  for  which  route 


discontinuities  can  be  stated  are  included.  The  number  of  adjacent  waypoints  denotes 
the  search  depth. 

An  accumulative  cost  index  based  on  track  angle  deviation  between  adjacent 
waypoints,  and  compared  to  the  direct  track  between  start/begin  and  end  of  the  planned 
flight,  leads  to  the  selection  of  the  next  waypoint  on  the  route. 

If  the  algorithm  detects  a  significant  track  angle  change  for  the  waypoint 
sequence,  which  is  described  by  the  previous  waypoint,  the  actual  waypoint  and  the 
selected  optimal  next  waypoint,  it  will  try  to  connect  them  directly,  thus  deleting  the 
intermediate  waypoint  entry. 

The  underlying  search  strategy  may  be  classified  as  an  informed  best-first  search 
[19]  supplemented  by  heuristic  elements.  If  a  graphic  visualisation  of  the  process  is 
available,  the  sequentially  performed  search  and  waypoint  evaluation  can  be  easily 
understood. 


In  the  following,  two  examples  will  provide  some  explanation  on  the  search  process 
and  the  verification  procedure. 

Verification  examples 

Aircraft  route  planning  is  performed  under  a  number  of  different  conditions,  which 
refer  to  different  constraint  list  editor  commands  and  operational  aspects, 
respectively.  Presently,  twelve  different  scenarios  have  been  realised  to  investigate 
planning  algorithm  and  optimisation  criteria.  Two  of  them,  representing  actual  flight 
conditions  will  serve  as  examples. 

Figure  5  illustrates  the  current  route  of  the  aircraft,  with  the  upcoming  waypoint 
to  be  reached,  WPBEG  as  starting  point,  the  intermediate  waypoints  ABC,  WPDEF  and  GHI 
and  the  endpoint  LMN.  The  algorithm  attempts  to  connect  the  waypoints  ABC  and  GHI 
directly,  thus  creating  the  route  description  or  constraint  list,  respectively,  as 
WPBEG,  ABC,  mHI,,  LMN. 

A  direct  connection  as  shown  between  LMN  and  WPSTU  will  be  reject^  .,  because  the 
algorithm  has  knowledge  of  the  restricted  area. 


Example  A  -  Command  input:  END  WPSTU 


With  this  command  input  the  endpoint  attribute  of  LMN  is  overwritten.  Within  a 
constraint  list  just  one  'end'  is  allowed. 


Phase  1:  The  waypoints  of  the  current 

constraint  list  and  the  new  list  entry 
WPSTU  are  sorted  to  the  distance  from 
the  starting  point,  resulting  in  the 
sequence  WPBEG,  ABC,  GHI,  WPSTU,  LMN. 
The  last  waypoint  is  identified  as  far- 
off  and  deleted. 


Phase  2:  There  is  only  one  connection 

existing  between  WPBEG  and  ABC,  which 
then  is  chosen  as  optimal.  In  the  next 
step  several  alternatives  are 
identified:  ABC  -  WPDEF,  ABC  -  GHI,  ABC 
-  OPR  representing  the  first  layer  of 
the  search  tree,  and,  additionally, 
WPDEF  -  GHI,  GHI  -  WPSTU,  OPR  -  WPSTU 
m  case  of  two  layers  activated.  A 
criterion,  based  on  track  angle 
deviations  at  the  distinct  waypoints, 
compared  to  the  direct  track  between 
WPDEF  and  WPSTU,  is  applied  to  the 
different  alternatives,  selecting  the 
optimal  waypoint.  The  procedure  results 
m  the  optimal  waypoint  • equence  WPBEG, 
ABC,  OPR,  WPSTU. 


Example  B  -  Command  input ;  INSERT  G5 


O* 


/ 

/N.WWIO 


,  Floor#  5  •  Rout#  Planning 

Phase  1:  The  waypoints  GHI  and  WPSTU, 

defining  the  airway  m  the  specified 

region,  represent  new  entries.  The  waypoint  GHI  occurs  twice,  because  it  is 
already  part  of  the  active  constraint  list.  It  is  reduced  to  a  single  entry. 
Sorting  the  waypoints  to  the  distance  from  the  starting  point,  results  m  the 
initial  waypoint  list  WPBEG,  ABC,  GHI,  WPSTU,  LMN. 


Phase  2:  Compared  to  example  A,  an  identical  search  is  performed.  But  here,  it 

results  in  another  waypoint  sequence,  characterising  the  optimal  route.  It  is 
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defined  by  WPBEG,  ABC,  GHI,  LMN.  The  waypoint  WPSTU  is  identified  as  not  optimal 
and  will  be  deleted.  In  consequence,  the  command  input  by  the  pilot  is  ignored. 

A  message  is  generated  to  inform  the  pilot. 


Problems  to  be  solved 

A  command  input  VIA  G5  instead  of  INSERT  G5  -  example  B  -  forces  the  algorithm  to 
include  at  least  two  subsequent  waypoints  with  the  airway  attribute  G5.  The  result  will 
be  the  waypoint  sequence  WPBEG,  ABC,  OPR,  WPSTU,  GHI,  LMN.  Such  solutions,  far-off  the 
optimal  route,  result  in  failure  states  of  the  algorithm,  m  which  the  algorithm  needs 
assistance.  By  means  of  heuristics,,  i.e.  rules  of  thumb,  these  problems  can  be  resolved 
in  future. 

Besides  aspects  regarding  the  search  algorithm,  presently  a  distinction  is  made 
regarding  enroute  and  terminal  area  planning  task,  because  of  different  data 
representations  for  enroute  airways  and  terminal  area  routing  [10].  A  common  internal 
representation  has  not  been  realised  yet.  In  general,  the  route  planning  procedure  is 
identical  for  both  enroute  and  terminal  area. 


5.  Software  Prototype  Realisation 


The  implementation  of  the  aircraft  route  planning  algorithm  requires  the  simulation 
of  constraint  management,  database  and  man-machine-interface.  The  simulation  may  be 
reduced  to  those  functional  elements  directly  related  to  the  route  planning  task. 

Regarding  the  constraint  management  module,  the  command  set  of  the  constraint  list 
editor  as  well  as  some  basic  buffer  manipulation  commands  to  facilitate  parallel  work 
on  different  constraint  lists  had  to  be  realised.  The  central  element  of  this 
simulation  is  represented  by  the  route  planning  algorithm  and  some  basic  error-checking 
which  enables  the  algorithm  to  cope  properly  with  incorrect  data  input.  Additionally,  a 
navigation  database  had  to  be  installed  providing  data  access  for  the  constraint 
management  and  for  the  route  planning  task.  The  third  element  to  be  realised  is  the 
interface  between  man  and  machine.  The  first  lay-out  was  centred  on  the  present  CDU 
design  simulated  on  a  computer  terminal.  These  requirements  represent  the  initial 
system  design  approach  for  "rapid  prototyping".  Objectives  of  the  system  simulation 
were  the  development  and  investigation  of  the  route  planning  algorithm,  the  structure 
of  the  database  contents  and  the  interaction  between  the  different  elements. 


The  selection  of  software  tools 
was  strongly  influenced  by  the  route 
planning  algorithm,  which  required 
handling  of  list  structured  objects 
and  ;ecursive  application  of 
procedures.  To  tread  new  paths,  a 
Common  Lisp  Programming  Environment 
on  a  PC/AT  was  chosen  for 

realisation.  £igur^ _ 6  illustrates 

the  current  system.  Because  of 
problems  regarding  accuracy  and 
general  run-time  in  the  initial 
development  phase,  the  calculation 
of  coordinates,  distances  and  tracks 
was  realised  with  a  Pascal  program. 
Data  exchange  between  Lisp  and 
Pascal  program  was  provided  by  a  PC- 
Assembler  routine.  The  map  display 
(plan  mode)  was  implemented  on  an 
IRIS  workstation  with  a  program 
written  in  C. 


PC-AT 
PERSONAL 
COMPUTER  AT 


WORKSTATION 

SILICON 

GRAPTACS 


Figure  6  -  Current  Realisation  of  Constraint  Management  Module 
Currently  the  system  is  and  Route  Planning  Mgorlthm 

reorganised  because  of  run-time  and 
data  storage  problems.  Finally,  the 

programs  written  m  Lisp  will  be  transferred  from  the  PC  to  a  workstation,,  with  the 
man-machine-interface  module  realised  as  a  separate  software  module. 
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Abstract 

Planning  systems  for  autonomous  and  semi-autonomous  vehicles  can  be  viewed  as  performing  very  high  level 
guidance  and  control  functions.  The  structure  of  hierarchical  planning  systems  parallels  that  of  the  management  and 
decision-making  structure  of  many  organizations  where  long-term,  less  detailed  skeletal  plans  are  developed  at  the 
highest  levels  of  the  hierarchy,  and  near-term,  fully  detailed  plans  are  developed  and  then  implemented  by  the  lower 
levels.  Mission  planning  system:  can  be  decomposed  into  planning  algorithms  and  planning  management  functions. 
Together,  they  provide  the  onboai  d,  intelligent  decision-making  required  to  plan  and  execute  the  nominal  mission  and 
modify  mission  activities  in  response  to  unforeseen  events,  with  little  or  no  reliance  on  human  intervention.  Planning 
Algorithms  produce  plans  of  actions  that  best  achieve  stated  mission  objectives  within  specified  mission  constraints  for  a 
given  mission  environment.  Planning  Management  Functions  control  the  execution  of  plans,  monitor  the  progress  of  the 
plans  currently  being  executed,  dec'de  if  replanning  is  necessary  and,  if  so,  define  the  revised  planning  problem  to  be 
solved  by  the  planning  algorithms.  In  this  paper,  an  architecture  for  a  mission  planning  system  is  presented  with 
descriptions  of  the  associated  planning  algorithms  and  planning  management  functions.  Implementation  considerations 
for  architectures  in  that  class  are  discussed,  and  an  approach  to  stochastic  modeling  of  the  planning  process  implied  by 
such  architectures  is  proposed. 


1  Introduction 


Planning  algorithms  and  planning  management  functions  together  provide  the  onboard,  intelligent  decision-making 
that  allows  an  autonomous  vehicle  to  plan  and  execute  its  mission  and  to  modify  its  activities  in  response  to  unforeseen 
events,  with  little  or  no  reliance  on  human  intervention.  Given  a  well-defined  planning  problem  (i.e.,  objectives, 
constraints  and  knowledge  of  the  mission  environment)  Planning  Algorithms  produce  a  plan  of  actions  that  best  achieves 
stated  mission  objectives  within  specified  constraints  for  a  given  mission  environment.  Planning  Management  Functions * 
monitor  the  progress  of  the  plans  currently  being  executed  to  decide  if  replanning  is  necessary  and,  if  so,  define  the 
revised  planning  problem  to  be  solved  by  the  planning  algorithms.  In  effect,  planning  management  functions  manage 
both  the  planning  process  and  the  execution  of  plans,  acting  as  the  interface  between  the  mission  planning  algorithms  and 
other  onboard  systems  such  as  fault  detection  and  isolation,  redundancy  management,  sensing  and  image  recognition 
systems,  navigation  and  control.  Much  of  the  effort  invested  in  automation  for  autonomous  vehicles  during  the  past 
decade  has  been  in  the  development  of  planning  algorithms  for  a  variety  of  mission  applications.  In  contrast,  little  effort 
has  been  made  toward  formalizing  the  development  of  the  associated  planning  management  functions  that  are  a  pre¬ 
requisite  to  successfully  fielding  planning  algorithms  in  operational  autonomous  and  semi-autonomous  vehicles 


The  complexity  of  mission  planning  and  planning  management  problems  has  led  to  the  use  of  hierarchical 
decompositions  to  make  those  problems  manageable  [lj.  At  the  C.S.  Draper  Laboratory,  hierarchical  planning  and 
scheduling  architectures  have  been  developed  in  several  problem  areas  including:  layered  control  architectures  for 
autonomous  submersibles  (21;  hierarchical  planning  management  architectures  for  autonomous  aircraft  [3];  and 
hierarchical  planning  and  scheduling  architectures  for  railroad  traffic  control  14).  For  each  of  these  activities  the  design 
objective  has  been  to  develop  an  architecture  which  allows  the  complex  planning  and  scheduling  problem  to  be 
ecomposed  into  manageable  pieces.  The  architecture  must  support  the  flow  of  information  (commands,  resource 
quests,  and  status  reports)  throughout  the  system  that  is  required  to  support  the  orderly  execution  of  the  planning 
ocess.  Mission  and  trajectory  plans  are  developed  within  the  hierarchy  to  optimize  a  selected  objective  function  (e.g., 
nimize  lethality,  fuel  or  time  or  maximize  mission  accomplishment)  subject  to  constraints  that  must  be  satisfied  by  the 
ns  (e.g.,  allocations  on  mission  time,  survivability  or  weapons  use). 


Within  the  planning  system,  the  level  of  detail  to  which  mission  activities  must  be  planned  by  a  given  planning 
algorithm  is  determined  by  spatial  and  temporal  planning  horizons.  Over  a  short  planning  horizon  actions  must  be 
planned  in  great  detail.  In  contrast,  it  is  unnecessary,  even  finite,  to  plan  actions  that  lie  beyond  a  short  planning  horizon 
at  the  same  nigh  level  of  detail.  Thus,  there  is  a  natural  decomposition  of  the  planning  function  into  a  hierarchy  of  planners, 
wherein  the  planning  horizon  decreases  and  the  level  of  detail  at  which  actions  are  planned  increases  as  one  moves  from 
higher  to  lower  levels  of  the  hierarchy.  In  this  paper,  our  discussions  will  be  presented  in  the  context  of  an  aircraft 
mission  planning  problem.  For  that  example  problem,  at  the  highest  level  of  the  hierarchy  ( Mission  level)  skeletal  plans  of 
the  entire  mission  are  constructed.  At  intermediate  levels  (RoutelActivity  levels),  near-term  actions  that  are  consistent  with 
the  strategic  plan  are  planned  in  greater  detail.  Finally,  at  the  lowest  level  of  the  hierarchy  (Flight  Safety  level),  commands 
are  generated  for  vehicle  subsystems  (e.g.,  sensors,  vehicle  propulsion  and  control  systems)  that  execute  the  mission  and 
ensure  vehicle  safety.  Figure  1  illustrates  the  plans  generated  for  the  aircraft  example  by  a  hierarchy  consisting  of  three 
levels.  The  uppermost  segment  of  that  figure  depicts  the  potential  mission  objectives  set  forth  in  the  mission 
requirements  as  well  as  threat  regions  that  are  known  a  priori  and  zones  within  which  vehicle  travel  is  restricted.  Below 


1  Planning  Management  Function*  include  what  are  traditionary  referred  to  a*  MiMton  Management  Function*  The  term  planning  management  I a 
emp’oyed  to  focus  attention  on  their  role  in  managing  the  onboard  miaaion  planning 
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that  are  illustrated  the  plans  generated  by  the  three  levels  of  planning.  These  are  discussed  in  more  detail  in  succeeding 
sections 


Figure  1  Planning  Hierarchy  Example 

This  paper  addresses  the  design  and  development  of  hierarchical  planning  systems  for  autonomous  and  semi- 
autonomous  vehicles  with  an  emphasis  on  the  planning  management  functions.  Section  2  discusses  the  hierarchical 
decomposition  of  the  planning  problem  and  the  relationship  between  planning  and  planning  management.  Section  3 
presents  a  system  architecture  for  implementing  a  hicrarc!  ical  planning  system  and  defines  the  planning  management 
functions  that  must  be  performed  in  such.a  hierarchy.  Some  considerations  in  implementing  the  planning  algorithms  and 
planning  management  functions  are  addressed  in  Section  4.  Section  5  discusses  the  differences  between  the  process  of 
producing  a  plan  and  the  plan  produced  and  proposes  a  modelling  approach  for  the  dynamic  planning  process 
represented  by  the  proposed  hierarchical  decomposition.  Finalty,  Section  6  closes  the  paper  with  a  summary  and  some 
concluding  remarks. 

■  2  HtERAxafloti  Decomposition  of  the  Punninc  Problem 

j  The  hierarchical  approach  to  decision-making  decomposes  a  large  problem  into  a  number  of  separate  levels  or  sub- 

1  problems,  thereby  reducing  the  overall  complexity,  ideally  with  little  reduction  in  overall  system  performance.  The  top 

j  level  of  the  hierarchy  plans  the  entire  mission  using  coarse  or  abstract  models.  As  one  progresses  downward  within  the 

i  decision-making  hierarchy,  the  spatial  and  temporal  scope  of  the  decision-making  functions  decrease  and  the  level  of 

?  modelling  detail  increases.  The  selection  of  the  individual  subproblcms,  the  specification  of  their  local  performance 

•-  criteria  and  the  modeiiing  of  their  interactions  are  ail  part  of  an  integrated  approach  to  developing  a  hierarchical 

decomposition.  An  important  goal  in  this  selection  is  to  maintain  a  balance  in  the  complexity  of  decision-making  effort 
;■  across  altlevels  of  the  hierarchy. 
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The  planning  system  described  in  this  paper  consists  of  a  hierarchy  of  planners  which  collectively  cooperate  to  solve 
mission  planning  problems.  Each  planner  consists  of  a  number  of  components:  planner-specific  data,  models,  reasoning 
mechanisms  (algorithms)  and  control  mechanisms  (management  functions).  The  levels  of  abstraction  of  each  of  these 
planner  components  vary  from  level  to  level  within  the  hierarchy,  but  must  be  self-consistent  at  any  given  level.  The 
control  mechanisms  not  only  control  the  planning  algorithms  at  each  level  of  the  hierarchy  but  also  control  the  flow  of 
information  and  control  among  the  various  planners  in  the  hierarchy.  This  ensures  that  the  planning  process  proceeds  in 
an  orderly  fashion.  Thus,  each  planner  within  the  planning  system  (i.e.,  the  hierarchy  of  planners)  provides  planner- 
specific  outputs  to  and  accepts  planner-specific  inputs  from  other  planners  within  the  planning  hierarchy. 

2.1  Planning  vs.  Management 

As  described  above,  there  are  two  principal  functional  elements  contained  within  each  level  o£  a  planning  system 
hierarchy:  planning  algorithms  and  planning  management  functions.  The  planning  management  functions  are 
responsible  for  three  classes  of  control  subfunctions:  controlling  plan  execution,  managing  interactions  among  adjacent 
levels  of  the  planning  hierarchy  and  implementing  replanning  decision  logic.  The  planning  algorithms  are  responsible 
for  developing  plans  whenever  the  planning  management  functions  have  decided  that  replanning  is  appropriate.  The 
development  of  individual  planning  algorithms  is  addressed  elsewhere  (see,  for  example  15,61)  and  is  outside  of  the  scope 
of  this  paper:  However,  the  interfaces  between  the  planning  algorithms  and  planning  management  functions  and  their 
mutual  interactions  are  addressed  here.  Each  of  the  planning  management  subfunctions  is  described  in  Section  3.  The 
following  basic  questions  must  be  answered  when  designing  a  decision  hierarchy: 

•  How  many  levels  are  required? 

•  What  should  their  temporal  horizons  be? 

•  How  should  decision-making  be  partitioned  within  a  level? 

•  What  constraints  and  objectives  should  be  passed  from  level  to  level? 

•  What  happens  when  any  given  decision-making  unit  cannot  meet  its  constraints? 

There  are  no  well  founded,  systematic  procedures  for  addressing  these  design  questions,  although  research  continues 
in  an  attempt  to  synthesize  one.  Empirical  evidence  suggests  that  system  performance  is  not  overly  sensitive  to  the  gross 
structural  form  of  a  hierarchy,  e.g.,  the  number  of  levels  and  how  they  are  partitioned.  The  difficulty  lies  in  the  design  of 
the  planning  and  management  functions  that  are  embedded  within  a  hierarchy.  These  design  issues  are  the  focus  of  this 
paper. 

2.2  Hierarchical  Decomposition  of  Automated  Mission  Planning 

To  make  our  discussions  concrete,  a  specific  hierarchical  decomposition  of  the  vehicle  mission  planning  problem  has 
been  selected.  The  selected  planning  system  hierarchy  has  three  levels:  the  mission  level,  the  route/activity  level  and  the 
flight  safety  level  (see  Figure  1).  The  nature  of  the  plans  and  the  planning  algorithms  that  are  used  to  generate  plans  at  each 
of  these  levels  are  described  below.  The  planning  management  functions  that  manage  the  execution  of  those  planning 
algorithms  are  also  described  in  a  subsequent  section. 

Two  types  of  waypoints  are  referred  to  in  the  discussions  of  planning  algorithms:  mission  waypoints  and 
intermediate  waypoints.  Mission  waypoints  are  associated  with  mission  objectives  and  are  specified  as  inputs  to  the 
planning  system.  Intermediate  waypoints  are  waypoints  that  are  generated  at  the  discretion  of  the  planning  algorithms. 
They  are  used  to  define  the  trajectories  between  selected  mission  waypoints.  Intermediate  waypoints  can  be  added  to  or 
deleted  from  the  mission  plan  by  the  planning  algorithms. 

2.2.1  Mission  Planning  Level 

At  the  mission  planning  level,  the  selection  and  ordering  of  a  subset  of  mission  waypoints  that  make  up  the  nominal 
mission  plan  and  the  definition  of  the  trajectories  at  a  low  level  of  resolution  between  the  mission  waypoints  [5,6)  are 
performed.  The  mission  planning  problem  is  decomposed  into  a  high  level  trajectory  planning  problem  and  a  goalpoint 
planning  problem,  each  with  an  associated  planning  algorithm  A  by-product  of  the  plans  generated  at  the  mission 
planning  level  is  a  resource  allocation  spanning  the  complete  mission  that  is  consistent  with  the  overall  mission  resource 
budget. 

2.2.1.1  High  Level  Trajectory  Planning  Algorithm 

The  high  level  trajectory  planning  algorithm  generates  near-optimal  (within  specified  constraints  on  time,  energy  and 
survivability)  trajectory  segments  between  mission  waypoints.  In  a  premission  planning  mode,  the  high  level  trajectory 
planning  algorithm  generates  a  set  of  trajectory  segments  linking  all  pairs  of  mission  waypoints.  During  a  mission,  some 
or  all  of  these  trajectory  segment  s  may  be  updated  at  the  discretion  of  the  planning  management  functions  to 
accommodate  changes  in  the  expected  mission  environment  (e.g.,  weather  and  threat',).  In  our  implementation,  trajectory 
segments  are  generated  by  a  modified  A*  search  over  a  predefined  set  of  candidate  mission  waypoints  (6).  The  A*  search 
methodology  has  been  selected  because  it  provides  a  computationally  efficient  mechanism  for  making  trade-offs  between 
solution  optimality  and  search  time.  In  generating  trajectory  segment  s  between  mission  waypoints,  the  A*  search 
employs  lethality  models  for  anticipated  threat  encounters  and  energy  consumption  models  that  account  for  vehicle 
airspeed,  altitude,  altitude  rate  and  winds.  The  high  level  trajectory  planning  algorithm  accommodates  (1)  moving  air 
traffic  (multiple  vehicles),  (2)  winds  and  weather  systems,  (3)  ground-to-air  threats,  (4)  specific  criteria  for  waypoint 
capture  and  (5)  corridors  bounding  trajectory  segments  between  specified  mission  waypoint  pairs.  The  trajectory 
segments,  together  with  estimates  of  the  costs  that  would  be  incurred  in  pursuing  them,  are  stored  in  a  trajectory  buffer 
for  subsequent  use  by  the  goalpoint  planning  algorithm. 

The  A*  search  methodology  makes  it  possible  to  obtain  multiple  solutions  (i.e.,  multiple  segments  between  a  given 
pair  of  mission  waypoints),  each  corresponding  to  a  different  optimization  criterion  and/or  set  of  constraints.  This 
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provides  the  goalpoint  planning  algorithm  described  below  with  a  set  of  trajectory  segments  that  trade  off  time,  energy 
and  lethality  in  different  ways. 

221.2  Goalpoint  Planning  Algorithm 

The  function  of  goalpoint  planning  is  to  use  the  high  level  trajectory  segments  and  their  associated  cost  (resource 
usage)  information  to  construct  a  mission  plan.  This  is  done  by  selecting  ar.d  ordering  the  subset  of  mission  waypoints 
(objectives)  and  associated  trajectory  segments  that  maximizes  the  utility  of  the  mission  plan  under  specified  constraints. 
In  our  implementation,  the  goalpoint  planning  algorithm  is  initialized  with  a  seed  plan,  typically  a  degenerate  plan 
consisting  only  of  the  origin  and  final  destination  of  the  vehicle.  This  plan  is  iteratively  improved  through  a  number  of 
modification  cycles  15,6).  During  these  modification  cycles,  the  best  plan  found  during  all  previous  cycles  is  stored  in  a 
best  plan  buffer.  Within  each  cycle,  the  plan  representing  the  initial  condition  for  that  cycle  is  modified  in  accordance  with 
a  set  of  heuristics.  The  output  of  this  process,  the  mission  plan  consisting  of  the  selected  mission  waypoints  and 
associated  high  level  trajectories,  is  placed  in  the  flight  plan  buffer,  which  contains  the  plans  that  are  generated  at  all  levels 
of  the  hierarchy. 

2.2.2  Route/Activity  Level 

The  route/activity  level  of  *he  hierarchy  is  responsible  for  filling  in  details  in  the  plan  generated  at  the  mission  level. 
Two  classes  of  plans  are  generated  at  this  level:  more  detailed  routes  between  the  intermediate  waypoints  that  are 
specified  in  the  the  high  level  trajectory  plan  segments  and  activity  plans  required  to  accomplish  the  mission  objectives 
associated  with  each  of  the  mission  waypoints  specified  by  the  goalpoint  plans.  The  objectives,  resource  allocations, 
timelines  etc.  for  generating  these  route  and  activity  plans  are  all  contained  in  the  plan  generated  at  the  mission  levrl. 
These  plans  have  a  shorter  spatial  and  temporal  horizon  than  the  plans  generated  at  the  mission  level.  In  order  to 
perform  planning  at  this  level,  the  mission  environment  must  be  described  at  a  higher  level  of  detail  than  is  required  at 
the  mission  level.  A  more  detailed  description  of  the  mission  environment  is  obtained  by  fusing  onboard  sensor  data  with 
a  priori  map  data. 

A  brief  description  of  a  route  planning  algorithm  is  given  below.  Since  the  nature  of  the  activity  planning  algorithms 
will  vary  significantly  as  a  function  of  the  designated  mission  objective  (e.g.,  reconnaissance,  supply,  ground  attack, 
interdiction,  etc.),  no  activity  planning  algorithm  is  described  here. 

22.2.1  Route  Planning  Algorithm 

The  primary  function  of  route  planning  at  this  level  of  the  hierarchy  is  to  plan  the  vehicle’s  flight  path  at  a  higher 
level  of  detail  than  is  provided  by  the  intermediate  waypoints  of  the  high  level  trajectory  segments  specified  at  the 
mission  planning  level.  The  route  planning  algorithm  produces  detailed  plans,  consisting  of  a  set  of  intermediate 
waypoints  that  are  consistent  with  both  the  resource  allocations  established  by  the  mission  plan  and  the  vehicle's 
performance  envelope.  The  inputs  to  the  route  planning  algorithm  are  the  mission  plans  stored  in  the  flight  plan  buffer 
and  the  current  best  estimate  of  the  state  of  the  mission  environment  supplied  by  the  information  fusion  function.  The 
output  consists  of  route  plans  connecting  a  pairs  of  mission  waypoints  or  intermediate  waypoints  defined  by  the  high 
level  trajectory  plan.  The  route/activity  planning  algorithms2  are  required  to  generate  plans  that:  (1)  adhere  to  flight 
corridors  and  mission  specifications,  (2)  avoid  local  storm  systems  and  adverse  winds,  (3)  avoid  collisions  with  local  air 
traffic,  (4)  avoid  collisions  with  the  ground,  (5)  account  for  vehicle  system  failures  or  damage  and  (6)  specify  the 
utilization  of  sensor  and  payload  systems. 

In  our  implementation,  route  plan  generation  is  accomplished  by  a  near-optimal  A*  search  similar  to  that  used  by  the 
high  level  trajectory  planning  algorithm  of  the  mission  planner.  A  major  difference  between  the  functions  of  the 
route/activity  and  mission  planners  is  the  information  used  by  each  in  generating  plans.  The  mission  planner  uses  a  priori 
and  communicated  map  data.  Because  the  route/activity  planners  generate  plans  incorporating  greater  detail  and  for 
shorter  temporal  horizons  than  the  mission  planner,  the  route  planning  algorithm  requires  both  real-time  sensor  data  and 
a  priori  data  for  generating  its  plans. 

223  Flight  Safety  Level 

During  the  execution  of  the  nominal  mission  plan,  events  that  have  not  been  anticipated  in  formulating  the  nominal 
mission  plan  and  which  threaten  the  safety  of  the  vehicle  may  occur.  The  planning  hierarchy  accommodates  these  events 
at  the  lowest  level  of  the  hierarchy  (see  Figure  2)  where  near-term  flight  safety  plans  are  generated.  These  plans  take 
precedence  over  any  other  plans  in  the  near  term. 

The  primary  function  of  the  flight  safety  planning  algorithm  is  to  provide  a  plan,  over  a  very  short  temporal  horizon, 
that  is  guaranteed  to  meet  survivability  requirements.  Secondary  emphasis  is  placed  on  time  and  fuel  optimization.  The 
fact  that  the  flight  safety  planning  algorithm  is  invoked  is  an  indication  that  mo-  :  reasoned  approaches  could  not  be  used 
to  obtain  a  plan  in  time  to  accommodate  the  current  safety  threatening  situation.  Inputs  to  the  flight  safety  planning 
algorithm  are  contained  in  the  flight  plan  buffer  (the  mission  plan,  route  plan  and  possibly  an  active  flight  safety  plan) 
and  the  current  best  estimate  of  the  state  of  the  local  mission  environment.  As  with  the  route  planning  algorithm,  the 
flight  safety  planning  algorithm  depends  heavily  on  fused  real-time  sensor  data  for  knowledge  about  the  state  of  the 
world.  The  detail  of  the  flight  safety  ,,'.an  is  comparable  to  or  higher  than  that  of  the  route  planning  level.  The  flight 
safety  planning  algorithm  produces  a  plan  that  is  executable  within  the  vehicle's  performance  envelope  and  that  ensures  a 
flight  path  that  is  within  an  acceptaisle  risk  of  vehicle  loss.  The  output  may  either  be  a  complete  flight  plan  to  the  next 
intermediate  '.vaypcir.t  generated  or  a  plan  whose  temporal  horizon  allows  sufficient  time  for  more  reasoned  planning  to 
regenerate  the  trajectory  to  the  next  intermediate  waypoint. 
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More  than  on*  algorithm  may  b*  required  to  meet  these  requirements 
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2-3  Planning  Management 

Whereas  planning  algorithms  generate  the  specific  sequences  of  actions  that  define  a  mission  plan,  planning 
management  functions  monitor  the  current  situation  for  events  that  may  require  a  replanning  response,  determine  the 
nature  of  that  response  by  defining  the  inputs  to  the  planning  algorithms,  and  control  the  interactions  among  the  several 
levels  of  planning.  The  planning  management  functions  play  the  role  of  an  executive  that: 

(1)  monitors  progress  relative  to  the  execution  of  the  current  mission  plan  and  the  vehicle's  ability  to  continue 
to  successfully  execute  the  current  plan, 

(2)  monitors  the  current  mission  situation  for  changes  in  the  state  of  the  vehicle  or  mission  environment  that 
may  necessitate  a  replanning  response 

(3)  monitors  any  changes  in  mission  objectives  or  constraints  that  may  necessitate  a  replanning  response 

(4)  determines  the  nature  of  the  replanning  response  by  selecting  the  appropriate  planning  algorithms  and 
defining  their  inputs 

(5)  controls  the  interactions  among  the  several  levels  of  the  planning  hierarchy 

Note  that  is  conceivable  to  implement  the  monitoring  functions  described  in  items  1-3  within  the  information  fusion 
function;  however,  this  is  undesirable  because  the  monitoring  functions  are  mission  related  and  go  beyond  the 
information  fusion  tasks  of  estimating  the  state  of  the  vehicle  and  its  operational  cn  vironment.  The  design  and  distributed 
implementation  of  these  planning  management  functions  are  outlined  below. 

3  System  Architecture 

The  multi-level,  distributed  architecture  depicted  in  Figures  2  and  3  has  been  developed  to  implement  the  planning 
algorithms  described  in  Section  2  and  their  associated  planning  management  functions.  Within  this  architecture,  the  total 
planning  management  function  is  partitioned  into'modular  sub-functior.s  and  distributed  throughout  the  hierarchy.  Each 
level  is  self-contained  and  has  its  own  set  of  planning  algorithms. 


Figure  2.  System  Architecture 

The  management  functions  at  each  level  make  decisions  based  on  three  sources  of  inputs:  the  mission  manager 
databases,  which  contain  information  about  the  state  of  the  aircraft,  the  state  of  the  environment  and  the  mission  objectives 
and  constraints;  the /hgftf  plan  buffer,  which  contains  the  current  plans  developed  at  each  level  of  the  planning  system;  and 
direct  communication  from  managers  at  adjacent  levels,  or,  in  the  case  of  the  mission  manager,  direct  communication  from 
a  planning  authority  specifying  changes  in  planning  requirements. 
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Figure  3.  Generic  Level  of  a  Planning  Hierarchy 

The  modular,  distributed  implementation  of  the  planning  management  functions  allows  easy  modification  of  one 
level  of  management  without  affecting  the  whole  system.  In  addition,  while  each  level  of  management  performs  similar 
functions  and  has  similar  interfaces,  the  modular  structure  allows  each  level  to  be  tailored  for  planning  activities  specific 
to  that  level. 

3.1  Management  Functions 

The  primary  role  of  the  planning  management  functions  is  to  control  the  planning  process  at  each  level  of  the 
hierarchy.  In  order  to  achieve  this  objective,  four  classes  of  management  functions  must  be  implemented  for  each  level  of 
the  hierarchy: 

(1)  Plan  execution  functions 

(2)  Plan  monitoring  functions 

(3)  Replanning  decision  functions 

(4)  Inter-level  coordination  functions. 

Each  of  these  is  discussed  in  detail  below. 

3.1.1  Plan  Execution  Functions 

Communication  between  the  planning  system  and  the  vehicle  guidance  and  control  functions,  sensors  and  payload 
is  made  through  the  planner-control  interface  block  in  Figure  2.  The  plans  developed  by  the  planning  algorithms  at  any  of 
the  planning  system  levels  contain  two  kinds  of  information.  One  kind  represents  descriptions  of  activities  that  are  to  be 
executed  by  the  guidance  and  control  functions,  sensors  or  payload.  The  other  kind  of  information  defines  requirements 
for  lower  levels  of  planning.  The  content  of  the  plan  defining  guidance  and  control,  sensor  or  payload  activities  must  be 
translated  by  the  planner-control  interface  into  formats  that  are  understood  by  the  subsystems.  Once  translated,  these 
activities  are  executed  by  the  appropriate  subsystem,  and  the  progress  of  that  execution  is  monitored.  Both  the  current 
progress  of  plan  execution  as  well  as  the  impact  that  the  current  progress  may  have  on  the  vehicle's  ability  to  successfully 
accomplish  activities  planned  for  the  future  are  monitored.  These  and  other  monitoring  functions  performed  by  planning 
management  are  discussed  below. 

3.1.2  Plan  Monitoring  Functions 
3.1.2.1  Monitoring  Plan  Execution 

Associated  with  each  mission  plan  is  a  measure  of  its  expected  accomplishment,  referred  to  here  as  its  utility,  and  its 
expected  profile  of  resource  utilization.  Thus,  the  progress  of  the  execution  of  the  current  plan  can  be  measured  along  two 
dimensions:  accomplishment  and  resource  utilization.  The  actual  level  of  mission  accomplishment  is  compared  with  the 
expected  level  of  mission  accomplishment.  The  actual  resource  utilization  is  compared  with  the  planned  profile.  If  the 
expected  accomplishment  or  the  actual  resource  utilization  differs  significantly  from  what  is  expected,  then  planning 
management  may  initiate  replanning.  If  the  expected  resource  utilization  exceeds  the  available  onboard  resources,  then 
replanning  will  produce  a  new  plan  that  can  be  accomplished  within  the  available  resources.  If  the  actual  resource 
utilization  is  lower  than  that  predicted  by  the  usage  profile,  then  replanning  might  be  in  /oked  in  an  attempt  to  exploit 
surplus  resources  to  produce  a  plan  of  greater  utility  than  that  of  the  current  mission  plan.  Fart  of  the  design  of  the 
planning  management  function  is  the  selection  of  the  decision  thresholds  that  are  used  in  the  comparisons  between  the 
actual  values  and  the  predicted  values.  Mechanisms  must  be  established  for  communicating  the  results  of  plan 
monitoring  to  provide  status  information  to  superior  levels  of  the  planning  hierarchy. 
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3.1.2J  Monitoring  Mission  Objectives  and  Constraints 

Any  changes  in  either  the  mission  objectives  (mission  waypoints)  or  explicit  mission  constraints  are  made  through  a 
user  interface  with  the  pilot /crew  in  a  piloted  vehicle  or  through  communications  from  a  higher  planning  authority  for  an 
autonomous  or  semi-autonomous  vehicle.  Changes  of  this  type  will  require  modifications  to  the  mission  requirements 
database  and  will  result  in  an  associated  request  to  replan.  These  changes  will  have  the  most  significant  impact  on 
planning  at  the  mission  level.  In  addition,  it  may  be  desirable  to  have  the  ability  to  externally  override  the  planning 
functions,  allowing  the  mission  plan  to  be  controlled  manually.  This  capability  will  be  useful  during  initial  flight 
demonstrations  of  autonomous  planners  since  it  provides  a  mechanism  for  manual  control  as  well  as  for  testing  specific 
flight  plans,  trajectories  and  vehicle  maneuvers.  This  point  is  further  elaborated  in  the  section  Levels  of  Autonomy. 

3.1.Z3  Monitoring  Vehicle  and  Environment  Status 

Planning  management  functions  are  required  in  order  to  monitor  the  status  of  both  the  vehicle  and  the  mission 
environment  in  order  to  detect  any  changes  that  may  affect  the  vehicle’s  ability  to  successfully  execute  the  current  mission 
plan.  This  monitoring  is  event-driven  and  is  based  on  information  received  from  the  information  fusion  function  and  its 
redundancy  management  sub-function.  In  our  implementation,  the  decision  to  initiate  replanning  in  the  face  of  events  of 
this  type  is  made  by  comparing  the  a  prion  plan  utility  with  the  value  of  expected  utility  conditioned  on  the  given  event. 

The  two  primary  classes  of  changes  in  vehicle  status  that  may  triggers  a  re-evaluation^  of  the  current  mission  plan 
are  failures  and  damage  to  the  vehicle.  For  redundant  systems,  a  failure  may  result  in  an  immediate  loss  of  performance 
as  well  as  in  reduced  subsystem  reliability  (c.g.,  one  channel  of  a  triplex  system  goes  down).  Here,  performance  refers  to 
the  vehicle’s  dynamic  capability  (i,e.,  performance  envelope),  sensing  capability  or  payload  system's  capability.  Reduced 
system  reliability  implies  a  lower  probability  of  having  the  capabilities  required  to  successfully  accomplish  future  mission 
objectives.  The  planning  management  functions  employ  an  onboard  system  reliability  model  to  predict  the  probability  of 
the  future  availability  of  specific  vehicle  subsystems  given  the  current  failure  status  of  those  subsystems.  These  reliability 
predictions  are  incorporated  into  the  computation  of  the  expected  level  of  accomplishment  of  candidate  mission  plans 
For  example,  a  decision  to  abort  a  mission  would  depend  significantly  on  the  number  of  redundant  elements  that  have 
been  lost  due  to  failures  or  damage  and  the  impact  of  the  failures  or  damage  on  system  reliability. 

Features  of  the  mission  environment  that  may  impact  the  expected  utility  of  the  current  plan  are  weather 
(winds/storms),  threats  ar.d  local  air  traffic  Timaly  information  regarding  these  features  of  the  mission  environment 
must  be  supplied  by  the  information  fusion  function. 

Once  a  significant  change  in  the  state  of  the  vehicle  or  environment  has  been  detected  by  the  planning  management 
functions,  one  of  several  events  may  occur:  (1 )  replanning  may  be  immediately  initiated,  (2)  the  plan  may  be  re-evaluated 
in  the  face  of  the  detected  change  and  on  the  basis  of  that  re-evaluation  a  decision  to  replan  can  be  made  or  (3)  replanning 
at  that  level  may  be  delayed  because  it  has  been  decided  that  a  lower  level  planner  should  first  respond  to  the  change  or  it 
is  has  been  observed  that  a  lower  level  planner  has  independently  begun  to  respond  to  that  change. 

3.1.2A  Onboard  Vehicle  Performance  Models 

Onboard  vehicle  performance  models  are  used  in  evaluating  candidate  plans  and  in  detecting  degraded  vehicle 
performance.  When  failures,  damage,  or  other  potential  performance  degrading  events  occur,  these  models  must  be 
updated.  Performance  monitoring  and  modeling  take  place  over  both  short  and  long  time  scales  and  at  various  levels  of 
abstraction.  The  updated  models  developed  over  long  time  scales  are  used  to  decide  whether  mission  replanning  may  be 
necessary;  and,  if  so,  the  updated  models  are  used  in  building  the  new  plan  and  allocating  resources  across  the  time  span 
of  that  new  plan.  Modeling  over  a  shorter  time  scale  is  used  in  emergency  situations  to  evaluate  the  immediate  impact  of 
unidentifiable  failures  on  the  performance  of  the  vehicle  and  its  subsystems  These  may  be  simple  input-output  models 
that  provide  approximate  response  of  the  vehicle  to  specific  commands. 

The  success  of  performance  modeling  requires  appropriate  interaction  with  mission  planning/management.  For 
example,  in  order  to  identify  or  update  models  of  vehicle  performance,  specific  vehicle  manei  vers  may  be  required  in 
order  to  identify  failed  actuators  or  to  "excite"  the  proper  vehicle  modes  to  identify  and  model  the  performance 
degradation  due  to  ineffective  aerosurfaccs. 

Qualitative  vs.  quantitative  models 

Examples  of  areas  where  performance  monitoring  and  updating  are  smportan'  from  the  point  of  view  of  both  fault 
tolerance  and  mission  planning/management  are: 

•  Vehicle  fuel  consumption  rate  as  a  function  of  flight  condition 

•  Vehicle  turn  /climb  /descend  capability 

•  Vehicle  speed  response 

•  Sensor  subsystems  performance 

•  Payload  subsystems  capability 

3.1.3  Replanning  Decision  Functions: 

Civen  inputs  from 

(1)  superior  levels  regarding  plan  requirements, 

(2)  subordinate  levels  regarding  planning  anu  plan  execution  status, 

(3)  the  planner  control  interface  regarding  plan  progress  and 

(4)  information  fusion  regarding  the  state  of  the  vehicle  and  the  mission  environment. 


evaluation  will  he  based  nnmxrllv  on  ihe  expected  utility  of  (he  current  plan  ©ven  Ihe  current  estimates  of  the  stale  of  the  vchide  and  environment. 


the  planning  management  functions  at  each  level  must  decide  whether  to  continue  to  pursue  the  current  plan  that  has 
been  developed  at  that  level  or  to  replan.  Given  that  a  decision  has  been  made  to  replan,  inputs  must  be  developed  for 
the  planning  algorithms.  In  effect,  those  inputs  define  the  problem  to  be  solved  by  the  planning  algorithms.  Included  in 
those  inputs  are  the  constraints  or  resource  allocations  that  must  be  satisfied  by  the  plan  (e.g ,  allocations  on  mission  time, 
fuel,  survivability,  and  weapons  use)  as  well  as  the  objective  function  that  should  be  optimized  in  developing  the  plan 
(e.g.,  minimize  lethality,  minimize  fuel,  minimize  time  and  maximize  effectiveness).  In  cases  for  which  there  are  multiple 
objectives,  the  management  functions  must  establish  the  relative  importance  among  those  objectives.  Finally,  given  that 
replanning  must  be  performed  in  real  time  (i.e.,  soon  enough),  the  management  functions  must  decide  how  much  time  is 
available  for  planning  and  how  the  onboard  computational  resources  should  be  allocated  in  generating  a  new  plan. 

3.13.1  Reconfiguration  /  Recovery  from  Failures 

Planning  management  plays  a  joint  role  with  redundancy  management  in  reconfiguration  and  recovery  following 
failures.  For  a  given  phase  of  a  mission,  the  best  configuration  of  the  aircraft’s  hardware  and  software  depends  on  the 
objectives  that  are  to  be  accomplished  during  that  phase.  In  addition,  recovery  from  failures  can  be  aided  by  the  lowest 
level  of  the  planning  hierarchy  in  that  the  lowest  level  specifies  short  term  flight  trajectories  requiring  only  the  most  stable 
flight  modes  immediately  following  a  significant  failure  event. 

In  the  event  of  a  failure,  the  primary  objective  of  redundancy  management  is  to  configure  the  vehicle  to  ensure 
stability  and  safety  of  flight.  Having  insured  that,  redundancy  management  provides  the  planning  system  with 
information  regarding  the  status  of  onboard  systems  and  the  range  of  vehicle  capabilities  that  are  possible  given  that 
status.  The  planning  systems  must  account  for  that  achievable  range  in  developing  plans.  Given  a  formulated  plan,  the 
planning  system  apprises  redundancy  management  of  the  associated  vehicle  capability  requirements,  and  redundancy 
management  configures  the  non-failed  vehicle  subsystems  in  a  manner  that  best  realizes  those  capabilities. 

For  aircraft  operations,  it  is  important  to  distinguish  between  two  types  of  failures  that  require  redundancy 
management .  One  type  consists  of  those  failures  lhat  impact  flight  safety.  These  types  of  failures  (e.g.,  actuator  failures, 
computer  failure)  represent  a  threat  to  immediate  safety  of  the  aircraft.  The  second  type  consists  of  those  failures  that 
impact  mission  effectiveness.  These  types  of  failures  (e.g.,  payload  failures,  sensor  failures)  do  not  pose  a  threat  to  the 
safety  of  the  aircraft  but  they  may  impact  the  mission.  For  safety  related  failures  redundancy  management  using 
standard  techniques.  However,  for  mission  related  failures,  redundancy  management  must  be  performed  in  the  context 
of  the  current  mission  situation  which  requires  interaction  with  the  planning  management  functions. 

3.1.3.2  Control  the  Generation  of  Plans 

Each  of  the  monitoring  functions  described  earlier  ultimately  provides  information  regarding  whether  or  not  to 
initiate  replanning.  Metaplanning  literally  translates  to  "planning  about  planning"  and  here  refers  to  the  way  in  which  the 
process  of  generating  a  new  plan  is  controlled.  Some  of  the  elements  of  metaplanning  .-elate  to  determining  the  level(s)  of 
the  hierarchy  at  which  replanning  should  be  performed,  decidingwhich  planning  algorithm  to  employ  at  a  give.,  level, 
and  establishing  the  time  allotted  to  generating  a  new  plan.  The  last"  section  of  the  paper  proposes  a  method  for 
modelling  the  planning  process  that  provides  insight  into  how  to  formulate  a  "control  law"  to  insures  stability  of  the 
planning  process. 

Planning  management  must  be  able  to  distinguish  between  small  departures  from  the  current  mission  plan  that  can 
be  corrected  by  making  small  perturbations  to  that  plan  (e.g.,  changes  in  commanded  airspeed)  at  the  route  planning  level 
and  significant  departures  that  can  only  be  corrected  at  the  highest  level  of  the  planning  hierarchy.  For  example, 
metaplanning  may  initially  restrict  the  mission  planning  level  by  icquesting  that  it  first  attempt  to  resolve  any  problems 
through  modifications  to  high  level  trajectory  plans  before  attempting  complete  replanning  of  the  mission  waypoint 
selection  and  ordering.  Discriminating  between  small  and  significant  departures  from  plans  generated  at  all  levels  of  the 
hierarchy  is  a  principal  metaplanning  function.  For  small  departures,  the  planning  algorithms  may  be  directed  to  initiate 
replanning  using  the  current  plan,  potentially  reducing  the  time  required  to  find  a  good  new  plan.  For  large  departures, 
replanning  may  begin  with  a  default  "empty"  plan,  requiring  the  new  plan  to  be  completely  rebuilt. 

3.1.4  Inter-Level  Coordination  Functions 

In  a  hierarchical  system,  higher  levels  provide  guidance  to  lower  levels.  In  the  planning  system,  the  plans  generated 
at  higher  levels  contain  both  constraints  and  objectives  that  serve  as  requirements  for  planning  at  lower  levels  (see  Figure 
2).  Hanning  management  is  responsible  for  translating  that  information  into  requirements  for  lower  levels.  Conversely, 
each  subordinate  level  receives  requirements  from  its  superior,  and  those  requirements  must  be  properly  interpreted  for 
input  to  the  planning  algorithms  at  the  subordinate  level.  In  addition,  each  subordinate  level  must  provide  information  to 
its  superior  regarding  the  status  of  the  execution  of  the  current  plans  generated  at  that  subordinate  level.  The  primary 
flow  of  information  within  the  hierarchy  planning  is  that  requirements  drive  the  planning  process  from  higher  levels  to 
lower  levels,  while  the  changes  in  the  operational  environment  drives  the  planning  process  from  lower  levels  to  higher 
levels. 

3.2  Implications  for  the  Design  of  Planning  Algorithms 

3.2.1  Isolating  the  Planning  Algorithms 

One  of  the  primary  objectives  of  the  design  of  the  architecture  depicted  in  Figures  2  ai.d  3  has  been  to  make  a  clear 
distinction  between  reasoning  and  control  mechanisms,  i.e.,  between  planning  and  management  functions.  The 
architecture  has  been  defined  in  a  manner  that  provides  clean  interfaces  -tween  the  planning  algorithms  and  the 
planning  management  functions.  The  major  benefit  of  this  approach  is  t,  it  simplifies  the  already  difficult  task  of 
designing  the  planning  algorithms  by  ensuring  that  management  issues  need  not  be  addressed  by  the  planning 
algorithms  themselves.  In  addition  the  planning  algorithms  are  packaged  in  a  modular  fashion  that  facilitates  the 
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insertion  of  now  algorithms  and  allows  one  planner  to  call  another  (see  Section  4)  without  consideration  of  potential 
management  side  effects. 


3.2.2  Time  to  Plan 

In  order  for  the  hierarchy  to  perform  effectively  in  real  time,  the  time  allocated  to  a  given  planning  algorithm  for  a 
specific  planning  task  must  be  established  by  the  planning  management  functions.  Constraints  on  time  to  plan  may  be 
defined  at  a  given  level  or  may  be  passed  down  from  a  higher  level.  In  addition  to  the  objectives  and  resource  constraints 
that  define  a  planning  problem,  each  planning  algorithm  also  requires  an  initial  state  as  an  input.  Indeed,  every  plan 
must  begin  at  a  point  in  both  space  and  time  that  is  reflected  in  its  initial  condition.  Choosing  that  point  in  the  future 
implicitly  bounds  the  time  available  to  plan.  Thus,  the  planning  management  functions  at  each  level  are  responsible  for 
coordinating  the  choice  of  an  initial  condition  on  the  planning  process  with  the  planners  at  other  levels.  Furthermore,  the 
planning  management  functions  must  decide  which  planning  algorithms  may  be  most  appropriate  given  the  available 
time  to  plan.  Establishing  the  time  to  plan  and  associated  planning  initial  conditions  is  a  non-trivial  decision-making 
problem,  but  one  that  is  crucial  to  the  overall  performance  of  the  planning  system. 


3.3  Levels  of  Autonomy 

The  level  of  autonomy  of  an  automated  planning  system  can  be  measured  in  terms  of  the  frequency  of  operator 
interaction  with  that  system  and  the  complexity  of  those  interactions.  The  more  frequent  and  complex  the  interactions, 
the  lower  the  level  of  autonomy.  A  vehicle  capable  of  operating  at  several  levels  of  autonomy  provides  an  effective 
mechanism  for  testing  and  evaluating  the  performance  of  autonomous  planning  and  decision-making  functions  without 
incurring  the  risks  of  completely  autonomous  operations.  It  also  provides  the  capability  for  initially  operating  an 
autonomous  vehicle  at  a  low  level  of  autonomy,  while  facilitating  a  gradual  transition  to  the  ultimate  objective  of 
complete  autonomy.  The  planning  management  functions  serve  as  the  interface  between  the  onboard,  automated  systems 
and  the  human  operator  (sec  Figure  4). 

In  order  to  design  planning  management  functions  that  permit  operations  at  multiple  levels  of  autonomy,  it  is  useful 
to  partition  the  planning  management  functions  in  a  way  that  is  natural  to  human  decision-makers.  The  execution, 
monitoring  ar.d  replanning  decision-making  functions  described  in  the  preceding  sections  represent  such  a  partitioning 
The  information  flowing  into  those  partitions  and  the  types  of  decisions  they  produce  as  outputs  will  be  the  same  whether 
the  decisions  are  made  autonomously  or  by  a  human.  In  the  case  of  the  operator-inahe-loop,  the  presentation  of  the  input 
information  to  the  human  decision-maker  is  provided  through  an  appropriate  user-interface.  Partitioning  the  decision¬ 
making  in  this  manner  provides  the  additional  benefit  of  making  the  autonomous  decision  logic  easier  to  test,  debug  and 
evaluate,  since  the  human  tester  can  participate  directly  in  the  tests  via  the  interface  designed  for  person-in-the-loop 
operations. 


The  design  of  the  planning  management  functions  should  be  structured  so  as  to  most  easily  accommodate  the  three 
potential  modes  of  operation  describ'd  above:  autonomous,  operator-in-the-loop  and  test4.  The  hierarchy  shown  in 
Figure  2  shows  a  planning  authority  interacting  with  the  planning  system  only  at  the  highest  level.  In  general,  interactions 
may  be  required  at  any  of  the  levels  shown  in  Figure  4,  especially  during  testing  of  the  system.  The  hierarchy  and  its 
planning  management  functions  should  be  designed  from  the  outset  to  accommodate  the  expected  variety  of  interactions 
and  modes  of  operation. 


Mission  Level 


Route  /  Activity 
Level 


Flight  Safely 
Level 


Figure  4  Person  in  the  Loop 


an  operator-ln-lh  e-loop  test  mode  will  be  required  even  for  autonomous  systems 
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4  Implementation  Considerations 

The  preceding  has  focused  on  the  requirements  for  the  planning  management  functions.  In  this  section  we  discuss 
important  issues  that  must  be  considered  when  implementing  the  planning  management  functions  in  a  hierarchical 
planning  system. 

Below  we  describe  a  standardized  software  framework  within  which  the  planning  management  functions  and 
planning  algorithms  at  each  level  of  the  planning  hierarchy  can  be  implemented.  Establishing  such  a  framework  will  not 
only  simplify  the  design  process,  but  will  also  make  it  easier  to  expand  the  system  design  at  some  future  time.  For 
example,  for  the  aircraft  planning  problem  discussed  here,  we  have  identified  a  hierarchy  containing  two  levels  of 
trajectory  planning  (referred  to  here  as  high  level  trajectory  planning  and  route  planning).  If  in  the  future  it  were  decided 
that  three  levels  of  trajectory  planning  were  preferred,  the  third  level  could  be  added  with  much  less  effort  if  a  standard 
framework  is  used  for  each  level. 

The  standardized  implementation  described  here  for  a  particular  level  of  the  hierarchy  contains  three  basic  functions 
(see  Figure  5): 

•  A  Manager :  The  component  which  contains  all  of  the  planning  management  functions  discussed  previously  in  the 
paper. 

•  One  or  More  Planners :  The  planners  generate  solutions  to  planning  problems  that  are  defined  by  the  manager 
and/or  other  planners. 

•  A  Manager/Planner  Interface-.  This  component  of  the  framework  provides  a  mechanism  for  controlling  the  flow  of 
information  between  management  functions  and  planning  functions  both  within  and  across  levels  of  the  hierarchy. 

The  details  of  the  design  of  the  software  within  each  component  need  not  be  identical  across  levels  of  the  hierarchy.  What 
is  important  is  to  standardize  the  information  and  control  flow  in  and  out  of  the  components. 


S«ft»  to 
Suptrior 


Tatkfrom 

Superior 


M 


imv&i 


SttlM 
from 
Info dor 


n 


Compltfrd  Job  lo 
higtior  Itvol  Ptvtnor 


Planning 


Took  to 

Intartor 


Planning  Raquast 
from  highor  kvl  Plannar 


.  MANAGER/ _ I 

rtri  PLANNER 
%  INTERFACE 


i»w» 


Queue 

Managei 


1M 


JOB 

Queue 


fee/ . . v . 


.  }  fx 

Complatad  Job''  -•  s'. 


jo 

gerJS^j 


Job 

Router 


\ 


Planning 
Job  ^ 


Comptnad  I 

Job 


Planning  Raqv+st 
to  io+ar  Sava!  Plannar 


rram? 


n 


Computed  Job  from 
k>*f  Plannar 


Figure  5.  A  Generic  Level  of  the  Mission  Planning  Hierarchy. 


4.1  The  Manager 

Consider  the  following  example.  The  mission  planner  creates  a  flight  plan  that  consists  of  a  number  of  mission 
waypoints  and  associated  high  level  trajectory  segments.  Prior  to  the  execution  of  a  new  trajectory  segment,  the  mission 
planning  level  passes  the  new  segment  to  the  route  level  manager  as  a  task.  The  task  includes  the  destination  waypoint 
and  the  operational  resource  allocations.  The  route  level  manager  directs  the  route  planning  algorithms  to  create  a 
detailed  route  plan,  to  execute  the  route  piatt,  monitoring  the  execution  of  that  pc  rtion  of  the  flight  plan,  and  to  replan  as 
the  situation  warrants.  In  situations  where  a  manager  cannot  produce  a  plan  that  satisfies  the  constraint  and  resources 
allocated  by  its  superior,  it  reports  back  to  the  higher  level  manager.  For  example,  if  the  route  level  manager  cannot 
produce  a  satisfactory  route  within  the  specified  resource  allocations,  it  reports  this  situation  back  to  the  mission  level 
manager.  The  mission  level  must  reallocate  the  available  resources  to  meet  the  current  shortfall  identified  at  the  route 
level  or  reformulate  the  planning  problem. 

When  the  manager  at  a  given  level  determines  that  the  effectiveness  of  the  plan  that  it  is  currently  executing  has  been 
reduced  cr  when  the  manager  identifies  an  opportunity  to  improve  the  effectiveness  of  the  current  plan,  the  manager 
generates  the  appropriate  job  and  sends  it  to  the  queue  manager. 

4.2  The  Manager/Planner  Interface 

To  support  the  flow  of  information  throughout  the  hierarchy,  it  is  desirable  to  standardize  the  structure  of  that 
information.  This  can  be  accomplished  by  employing  a  data  structure  referred  to  here  simply  as  a  'lob".  A  job  identifies  a 
planning  problem  that  some  element  of  the  hierarchy  (either  a  manager  or  another  planner)  has  defined.  A  job  is  a 
mechanism  for  packaging  a  planning  problem  into  a  standardized  form  that  can  be  communicated  throughout  the 
hierarchy.  Each  job  is  made  up  of  seven  elements; 
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•  ID:  A  unique  ID  that  allows  the  hierarchy  to  keep  track  of  the  different  jobs  in  the  system. 

•  Creator:  The  manager  or  planner  that  created  the  job.  This  information  is  used  by  the  job  router  to  know 
where  the  results  of  the  job  should  be  directed  when  it  is  completed. 

•  Planner:  The  planner  to  which  the  job  is  to  be  sent.  This  information  is  required  to  define  the  destination 
for  a  job  when  there  is  more  than  one  planner  within  a  level  of  the  hierarchy. 

•  Priority:  A  value  that  indicates  the  relative  importance  of  the  job. 

•  Start-Up  Time  The  time  the  job  was  created.  The  start-up  time,  together  with  the  job's  priority,  is  used  to 
determine  the  job's  location  in  the  job  queue. 

•  Problem  Description:  The  complete  set  of  inputs  to  the  planning  algorithm  including  objectives, 
constraints,  resource  importance  factors  and  the  time  available  to  plan. 

•  Problem  Solution :  The  place  holder  used  to  store  the  solution  to  the  planning  job  when  the  planner  has 
completed  its  task. 

•  Status:  The  status  of  the  job  (e.g.,  on  the  queue,  executing). 

Jobs  are  initiated  by  the  manager.  They  are  deposited  in  a  job  queue  within  the  Manager/Planner  Interface  according 
to  a  priority  and  time  spent  in  the  queue.  When  a  job  reaches  the  top  of  the  queue,  it  is  executed  by  the  planning 
algorithm  and  the  results  ( problem  solution )  are  sent  back  to  the  requesting  manager  or  planner  via  the  job  router.  At  any 
point  during  the  execution  of  a  job,  the  manager  can  direct  the  queue  manger  to  interrupt  or  terminate  the  job.  Planning 
algorithms  can  also  create  jobs  fot  planning  algorithms  at  lower  levels  of  the  hierarchy.  These  jobs  get  handled  in  an 
identical  manner  although  the  information  is  passed  across  levels  of  the  hierarchy  An  example  of  a  planning  algorithm 
generating  a  job  occurs  in  the  mission  planner.  In  order  to  determine  the  best  ordering  of  the  mission  waypoints,  the 
goalpoint  planning  algorithm  must  know  the  approximate  cost  of  flying  between  pairs  of  mission  waypoints.  It  obtains 
this  information  by  sending  a  job  to  the  high  level  trajectory  planning  algorithm. 

4.3  The  Planner 

The  system  architecture  depicted  in  Figure  2  has  been  developed  to  eliminate  the  need  for  planning  algorithms  to 
treat  management  issues.  One  concern  when  orchestrating  the  execution  of  the  planning  algorithms  is  the  need  to  balance 
the  computational  load  of  the  system  so  that  planning  is  completed  across  all  levels  "just  in  time".  Even  though  the 
management  functions  perform  the  command  and  control  associated  with  the  hierarchy,  it  is  expected  that  in  most  cases, 
the  bulk  of  the  computational  requirements  will  come  from  the  planning  algorithms.  It  is  therefore  particularly  important 
that  the  planning  algorithms  be  designed  to  perform  their  functions  within  a  specified  availability  of  the  computational 
resources  of  the  system.  One  method  of  achieving  this  goal  is  for  the  management  functions  to  impose  "time  to  plan" 
limitations  on  the  planning  algorithms  during  operation. 

5  Topic  for  Future  Research;  Modelinc  the  Planning  Process 

In  this  section,  we  propose  to  model  the  process  of  formulating  a  mission  plan  (cither  the  complete  initial  plan  or  an 
updated  plan  created  in  response  to  an  unexpected  event  during  mission  execution)  as  a  discrete  event  stochastic  process 
(/).  By  modelling  the  planning  process  in  this  way,  one  can  gain  insight  into  how  to  develop  the  control  law  for  such  a 
process,  that  control  law  being  embodied  in  the  planmng.managomont  decision-making  functions  described  earlier. 
Considerations  in  creating  such  a  model  are  discussed  below  in  the  context  of  the  initial  plan  generation  process. 
Generalization  to  the  replanning  process  is  discussed  at  the  end  of  the  section.  It  should  be  noted  that  we  only  introduce 
out  approach  to  modelling  the  planning  process  and  suggest  avenues  for  further  research  into  this  modelling  and  control 
problem. 

The  process  of  generating  an  initial  plan  begins  at  the  highest  level  of  the  hierarchy.  The  objectives  and  constraints 
(resource  allocations)  that  define  the  planning  problem  at  each  level  successively  "flow  down"  from  superior  to 
subordinate  levels.  Ideally,  each  subordinate  level  creates  a  plan  that  "best"  accomplishes  the  objectives  set  forth  by  the 
superior  within  the  allocated  resources  andjpecified  time-to-plan.  The  metric  (or  metrics)  by  which  the  "best"  plan  is 
measured  is  also  established  by  the  superior’.  This  provides  a  mechanism  to  insure  that  the  overall  measures  of  mission 
accomplishment  remain  consistent  across  planning  at  all  levels.  If  sufficient  resources  are  allocated  for  accomplishing  the 
objectives  defined  for  each  successive  subordinate  level,  then  planning  will  be  successful  on  the  first  try  at  every  level  and 
the  total  time  to  generate  a  complete  plan  (that  is,  across  all  levels  of  the  hierarchy)  is  a  deterministic  quantity  and  is  simply 
the  sum  of  the  times  required  for  generating  plans  across  all  levels.  Unfortunately,  since  each  superior  level  must  use 
models  to  "predict"  the  expected  resource  requirements  for  its  subordinated),  those  predictions  will  not  always  be 
sufficiently  accurate  to  ensure  this  ideal  solution.  Specifically,  it  will  often  happen  that  the  subordinate  at  some  level  will 
discover  that  the  resource  allocations  predicted  and  passed  down  by  its  superior  are  insufficient  to  create  a  feasible  plan, 
much  less  an  optimal  one. 

Of  course,  the  ideal  flow  down  solution  could  be  achieved  if  each  superior  level  always  provided  a  large  resource 
margin  relative  to  its  predictions.  Specifying  generous  resource  allocations, can  result  in  a  plan  of  reduced  overall  mission 
accomplishment  relative  to  the  maximum  accomplishment  possible  using  all  available  resources,.  Providing  too  little 
margin  in  resource  allocations  can  result  in  a  planning  deadlock  due  to  subordinates  being  unable  to  find  feasible  plans. 
As  illustrated  by  the  following  example,  the  tradeoff  between  generous  and  tight  resource  margins  is  one  that  can  be 
addressed  by  the  proposed  model  of  the  planning  process. 

Consider  two  cases  wherein  the  resource  allocations  provided  by  the  superior  are  inconsistent  with  the  plans  that  can 
be  developed  by  the  subordinate  using  higher  fidelity  models:  (I)  the  subordinate  requires  more  resources  for  a  feasible 
solution  and  (IB  a  good  plan  requires  far  fewer  resources  than  predicted  by  the  superior.  In  case  I,  the  subordinate  must 


5  At  all  levels,  secondary  measures  of  plan  merit  relating  to  resource  usage  should  also  be  considered  The  relative  weighting  of  one  type  of  resource  versus 
another  <e.g,  fuel  vs.  survivability)  is  also  established  by  the  supenor  level 
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send  a  request  back  up  to  the  superior  for  either  more  resources  or  a  reduction  in  the  scope  of  the  expected  objectives  to  be 
planned  for  at  the  subordinate  level.  Along  with  this  request  the  subordinate  may  also  send  the  best  plan  (or  plans)  that 
could  be  produced  for  the  stated  objectives  and  the  additional  resources  required  to  pursue  that  plan.  In  this  case,  the 
superior  must  decide  how  to  reallocate  resources  across  its  larger  (relative  to  the  subordinate)  temporal  and  spatial 
horizons,  possibly  relaxing  objectives,  in  order  to  make  subordinate  level  planning  feasible.  That  reallocation  implies  a 
change  in  the  superior's  plan,  potentially  abrogating  the  planning  effort  already  expended  at  other  levels  and  thereby 
delaying  the  time  to  complete  the  planning  process.  Alternatively,  the  superior  could  pass  the  problem  further  up  the 
hierarchy  to  have  the  resource  inconsistency  addressed  at  a  yet  higher  level.  Of  course,  the  higher  in  the  hierarchy  that  a 
problem  is  resolved,  the  greater  the  extent  of  lower  level  planning  effort  that  may  be  obviated,  further  increasing  delay  in 
completing  the  planning  process. 

In  case  II,  the  subordinate  returns  its  best  plan  along  with  an  indication  of  the  associated  resource  surplus.  The 
superior  may  choose  not  to  reallocate  the  surplus,  with  the  consequence  that  a  plan  of  potentially  higher  levels  of 
accomplishment  may  be  sacrificed  (other  considerations  regarding  resource  surpluses  are  discussed  later  in  this  section). 
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Figure  6  Planning  Process  State  Transition  Diagram 

One  element  of  the  stochastic  modelling  problem  is  that  of  approximating  the  impact  on  the  planning  process  of  the 
two  effects  of  resource  budgeting,  namely,  resource  margins  that  are  too  tight  can  lead  to  an  unstable  planning  process 
while  resource  margins  that  are  too  generous  can  lead  to  an  overall  plan  of  significantly  reduced  effectiveness.  Thus,  one 
of  the  models  that  must  be  developed  is  of  the  probability  that  a  subordinate  will  be  unable  to  create  a  feasible  plan  that 
satisfies  the  superior's  resource  margins.  In  the  simplified  planning  process  state  transition  diagram  in  Figure  6,  that 
probability  is  denoted  by  Pj,j(rmargin  i  j)  with  Level  i  as  the  superior  and  Level  j  as  the  subordinate  and  with  rmargin  i,j  a 
measure  of  the  resource  margin  provided  by  Level  i  for  Level  j's  objectives.  The  probability  distribution  for  -he  total  time 
to  complete  planning  is  computed  as  a  function  of  the  times  to  generate  a  plan  at  each  level  and  the  probabilities  of  the 
planning  process  transitioning  either  up  or  do  wn  in  the  hierarchy.  Clearly,  this  model  is  a  gross  simplification  in  that  the 
time  to  plan  at  any  level  is  specified  by  other  planning  management  functions  which  would  also  require  modelling. 
However,  even  simple  approximate  models  may  provide  significant  insight  into  the  impact  of  the  selection  of  the  resource 
margins.  Finally,  the  stability  of  the  overall  process  is  a  major  concern  and  it  is  anticipated  that  this  approach  to 
modelling  will  be  useful  in  assessing  stability  and  in  quantifying  stability  margins. 

In  addition  to  the  planning  process,  there  is  a  stochastic  element  to  plan  execution  that  lends  itself  to  modelling.  That 
is,  the  degree  to  which  a  plan  is  susceptible  to  failure  in  the  face  of  unforeseen  events  is  partially  a  function  of  resource 
margins.  That  is,  when  margins  are  tight  and  additional  resources  are  required  for  near-term  accommodation  of 
unforeseen  events,  the  long-term  plan  may  become  inviable  due  to  a  lack  of  sufficient  resources.  Again,  there  is  a  trade 
off  between  liberal  resource  margins  that  provide  the  reserves  required  to  absorb  the  impact  of  unexpected  events  and 
tight  resource  margins  that  are  set  in  order  to  allocate  as  much  of  the  available  resources  as  possible  to  accomplishment  of 
valued  objectives.  Replanning  is  initiated  when  the  plan  fails,  if  the  expected  rate  of  plan  failure  (the  rate  of  initiating 
replanning)  is  sufficiently  high,  then  there  may  be  insufficient  onboard  computational  resources  to  support  that  amount  of 
(replanning.  Thus,  this  additional  effect  of  resource  margins  must  also  be  accounted  for  in  the  design  of  the  planning 
management  functions. 

6  Summary  and  Conclusions 

An  architecture  for  mission  planning  systems  for  autonomous  or  semi-autonomous  vehicles  has  been  described.  That 
architecture  has  been  described  in  the  context  of  a  specific  planning  system  hierarchy  designed  for  aircraft  mission 
planning  problems.  However,  the  discussions  here  apply  equally  across  a  spectrum  of  space,  air,  land  and  undersea 
mission  planning  problems.  A  distinction  has  been  drawn  between  the  algorithms  that  create  plans  and  the  decision¬ 
making  functions  that  control  the  process  of  creating  and  executing  the  plans:  the  planning  management  functions.  A 
detailed  discussion  of  the  decision-making  functions  that  comprise  planning  management  has  been  presented  and  a 
standardized  framework  has  been  proposed  for  implementing  some  of  the  elements  of  the  planning  architecture 
associated  with  planning  management.  Finally,  an  initial  approach  to  developing  a  stochastic  model  of  the  planning 
process  has  been  proposed.  Models  of  this  type  will  be  useful  in  helping  to  design  the  decision-making  functions  that 
control  the  planning  process. 
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SUMMARY 

This  paper  describes  the  problems  and  issues  of  developing  a  tactical  mission  manager.  It  discusses  development  aspects  of  intelli¬ 
gent  real-time  avionics,  and  outlines  an  efficient  real-  time  Ai  methodology  and  implementation  for  the  development  of  the  intelligent 
systems.  It  also  outlines  advanced  software  development  techniques  and  provides  an  overview  of  related  Boeing  research  efforts. 

INTRODUCTION 

Tactical  combat  missions  are  becoming  more  complicated  due  to  improved  enemy  tactics,  equipment,  communications,  and  training. 
Advanced  aircraft  perfoimance,  and  sophisticated  subsystems  and  sensors  increase  aircrew  work  load.  The  missions  are  more  dynamic 
due  to  the  introduction  of  highly  mobile  threats,  real  time  intelligence  updates,  and  advanerd  battle  management  and  control  interfaces 
To  successfully  cany  out  a  modem  tactical  combat  mission  requires  the  aircrew  to  synthesize.,  filter,  comprehend,  end  respond  to  mass¬ 
es  of  sensor,  threat,  mission,  and  aircraft  systems  data  in  real  time.  Only  a  combat  aircraft  equipped  with  a  flexible,  real  time,  adaptive 
mission  planning  and  execution  capability  can  perform  effectively  in  this  environment 

The  automated  inflight  mission  manager  provides  real-time  mission  planning  and  replanning  capabilities,  facilitates  increased  air¬ 
crew  situation  awareness,  monitors  aircraft  and  mission  status,  analyzes  key  mission  situations  and  problems,  and  provides  the  aircrew 
with  intelligent  courses  of  action.  The  mission  manager  is  an  intelligent  avionics  system.  Intelligent  systems  differ  from  conven¬ 
tional  systems,  primarily,  because  they  must  make  their  own  decisions.  Conventional  avionics  systems  spend  most  of  their  time  pro¬ 
cessing  algorithms.  Intelligent  avionics  systems  spend  a  great  deal  of  effort  processing  complex  knowledge  or  logic  paths  These  logic 
paths  are  usually  based  on  expert  derived  experience  based  knowledge. 

A  great  deal  of  the  complexity  of  intelligent  software  is  contained  in  the  reasoning  and  decision  making  logic.  The  algorithms  still 
contain  the  bulk  of  the  code  and  perform  most  of  the  processing  functions. 

High  order  programming  languages  provide  effective  constructs  for  developing  algorithmic  computer  code.  However,  only  limited 
facilities  are  available  to  encode  the  complex  logr  definitions.  Intelligent  avionics  systems  must  process  large  amounts  of  logic  to 
make  their  decisions.  When  the  logic  can  not  be  broken  down  hierarchically,  software  complexity  is  greatly  increased.  The  code  be¬ 
comes  very  complex,  incomprehensible,  and  unmaintainable. 

Artificial  Intelligence  (Al)  knowledge  based  techniques  were  developed  to  allow  problem  domain  knowledge  to  be  acquired  from 
experts  and  efficiently  encoded  lor  automated  computer  processing,  reasoning,  and  decision  making.  This  allows  the  development  of 
system,  that  efficiently  encode  the  knowledge  obtained  from  experts,  and  provides  the  facilities  necessary  to  carry  out  the  automated 
reasoning  process.  The  problem  with  current  knowledge  based  systems  is  that  they  generally  do  not  function  on  avionics  computers,  in 
embedded  environments,  or  in  real  tune. 

The  embedded  avionics  environment  contains  some  specialized  characteristics  that  distinguish  it  from  many  conventional  AI  envi¬ 
ronments.  These  differences  make  it  necessary  to  develop  specialized  architectures  and  software  that  exploit  the  idiosyncrasies  of  em¬ 
bedded  systems.  When  exploited,  these  characteristics  allow  efficient  development  of  very  intelligent  system.!.  In  fact,  they  bound  the 
AI  problem,  allowing  processing  shortcuts  which  increase  efficiency  and  reduce  AI  processing  overhead. 

Developing  an  automated  inflight  mission  manager  requires  the  formulation  of  knowledge  based  software  development  techniques, 
methods,  and  tools.  Development  environments  must  be  built  that  will  allow  average  skill  level  computer  programmers  to  develop,  test, 
integrate,  and  validate  the  intelligent  software.  Run-time  environments  must  be  developed  that  provide  low  overhead,  real-time, 
knowledge  driven  software  processing.  These  systems  must  use  conventional  programming  methods,  practices,  engineers,  languages, 
and  facilities  to  create  hybrid  intelligent  AI  and  conventional  avionics  systems. 


AUTOMATED  INFLIGHT  MISSION  MANAGER 

The  objective  of  the  Automated  Inflight  Mission  Manager  is  to  enhance  the  attack  aircrait’s  capabilities  for  the  I990’s  and  beyond, 
emphasis  has  been  placed  on  the  ground  attack  role  of  tactical,  strategic  and  unmanned  aircraft  Enhanced  mission  capabilities  include: 
a  paperless  cockpit,  where  the  combat  mission  folder  is  contained  within  the  aircraft  avionics;  automated  control  of  an  enhanced  sensor 
suite  to  improve  target  search,  detection,  and  identirication;  intelligent  threat  assessment  and  countermeasure  selection  to  minimize  risk 
to  the  aircraft;  automated  mission  planning  and  replanning  to  provide  mission  adaptability  and  flexibility;  and  management  of  advanced 
controls  and  displays  to  minimize  aircrew  work  load  and  enhance  situation  awareness  The  inflight  mission  manager  internalizes  the 
hard  copy  mission  folder  used  by  aircrews  today,  and  represents  it  as  a  paperless  electronic  mission  folder  which  can  be  cnanged  and. 
updated  inflight  by  the  aircrew.  The  following  capabilities  are  required  of  the  automated  mission  manager: 
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1)  An  intelligent  sj  ;tem  to  synthesize,  prioritize,  process,  and  display  mission  events  in  real  time  to  the  aircrew.  The  system  will 
provide  intelligent  mission  problem  resolution  for  aircrew  decision  actions,  enhance  weapon  system  survivability,  and  increase  the 
probability  of  mission  success. 

1  2)  An  automated  inflight  capability  to  plan  and  replan  mission  activities  in  response  to  degraded  aircraft  performance,  unforeseen 

■  events  and  defenses,  and  changes  in:  targets,  threats,  routing,  timing  and  air  refueling. 

3)  An  interactive  system  that  incorporates  aircrew  and  communications  inputs,  resident  data  bases,  algorithms  and  sensor  data  for 
computing  and  optimizing  inflight  re-routing,  threat  avoidance  and  weapons  application.  The  system  recommended  actions  will  be  pre¬ 
sented  to  the  aircrew  in  a  time  critical  action  priority  sequence. 

,  Mission  manager  problem  inspection  reveals  that  mission  management  is  actually  a  collection  of  individual  management  problems. 

These  problems  are  separate  and  distinct  in  nature,  function,  and  task,  but  are  highly  dependent  on  each  other  for  information  and  con 
trol.  The  major  task  in  oudining  an  acceptable  approach  to  developing  a  mission  manager  is  defining  the  individual  management  prob¬ 
lems  and  their  inter-relaiionships. 

\ ' 

Individual  mission  management  problems  cover  a  wide  variety  of  functions  that  require  vastly  different  solution  processes.  Exam¬ 
ples  of  individual  problems  are:  checklist  management,  aircrew  interface,  mission  planning,  sensor  control,  threat  assessment,  counter¬ 
measure  selection,  and  aircraft  status  updating  and  monitoring.  Each  of  these  problems  is  unique  in  its  nature.  Some  can  be  solved  with 
standard  algorithmic  programming  techniques,  others  require  automated  reasoning  techniques  and  some  demand  combined  reasoning 
and  algorithmic  solutions. 

Control  of  multiple  mission  and  avionics  systems  and  processes  requires  an  intelligent  system  director.  The  system  director  is  an  in¬ 
telligent  controller  that  controls  and  focuses  system  processing  and  resources  that  by  scheduling  and  invoking  system  functions.  The 
system  director  handles  the  communications  between  the  individual  mission  manager  problem  solvers  and  accomplishes  two  major 
tasks.  See  figure  1. 

The  first  system  director  task  is  to  activate  the  mission  plan,  and  carry  out  the  mission.  The  mission  plan  is  the  contract  between  the 
aircrew  and  the  automated  mission  manager  system.  It  contains  all  information  needed  to  successfully  carry  out  the  mission.  This  is  by 
far  the  most  important  task  the  system  director  performs.  Even  on  a  perfect  mission  where  no  abnormal  events  occur  the  mission  man- 
\  ager  is  essential.  It  controls  aircrew  displays,  ensures  correct  procedures  are  known  and  followed,  and  monitors,  analyzes  and  filters 

mission  and  aircraft  status  to  provide  the  aircrew  with  the  best  possible  situation  awareness.  The  system  director  performs  this  task  by 
sequencing  through  the  mission  plan  and  activating  applicable  processes. 

The  second  system  director  task  is  to  analyze  the  global  world  state  and  look  for  anomalies  that  may  preclude  executing  the  mission 
as  planned.  This  involves  monitoring  the  aircraft,  weapons  and  mission  status.  The  system  director  may  invoke  mission  replanning, 
threat  avoidance,  or  other  mission  manager  processes  to  modify  the  mission  plan.  The  system  director  performs  this  task  by  looking 


•  The  inflight  mission  manager  is  a  hierarch'  jf  cooperating  systems 

•  System  director  controls  and  monitors  tht  .ndividual  systems 

•  Global  data  is  stored  in  the  blackboard .  ■  cture,  available  to  alt  systems 


FIGURE  1:  MISSION  MANAGER  ARCHITECTURE 

1 

*  tor  distinctive  mission  situations  which  are  passed  to  the  system  director  from  the  individual  mission  manager  component  processes, 

j  The  system  director  then  takes  the  appropriate  responses  by  scheduling  required  mission  manager  functions  to  be  executed. 

The  mission  manager  performs  crucial  mission  tasks  which  require  the  real-time  analysis  and  interpretation  of  numerous  dissimilar 
;  pieces  of  information.  It  performs  autonomous  or  semi-autonomous  reasoning  and  decision  making,  and  filters  vast  quantities  of  data 

;  to  reduce  operator  workload  and  to  ensure  that  correct  decisions  are  made  and  correct  actions  taken.  It  must  be  adaptable  to  changing 

*  requirements  both  during  mission  execution  and  over  the  entire  system  lifecycle.  This  is  especially  crucial  because  it  contains  laige 

«  amounts  of  heuristic  user  defined  logic. 
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INTELLIGENT  AVIONICS  SYSTEMS 

The  automated  inflight  mission  manager  is  an  example  of  an  intelligent  avionics  system.  Reasoning  and  decision  making  functions 
must  be  embedded  directly  in  it  to  provide  the  level  of  competence  necessary  to  accomplish  the  complex  missions.  The  system  must 
function  autonomously  or  semi-autonomously  and  should  reduce  not  increase  aircrew  workload. 

Intelligent  avionics  systems  differ  from  conventional  avionics  systems,  primarily,  because  they  must  make  decisions  based  on  user 
defined  logic  statements.  Conventional  systems  spend  most  of  their  time  processing  algorithms.  They  process  only  enough  logic  to 
drive  the  algorithms,  and  little  of  this  logic  is  very  complex. 

Conventional  avionics  software  is  complex,  difficult  to  understand  to  maintain.  A  contributing  factor  is  that  the  logic  and  algorithms 
are  woven  together  forming  complex  object  relations. 

Programs  that  contain  a  lot  of  user  defined  logic  or  knowledge  must  uc  flexible  and  adaptable  to  constant  change.  This  is  because 
their  problems  have  not  been  completely  defined.  They  contain  too  many  paths  to  enumerate  and  evaluate  all  of  them.  The  large  num¬ 
ber  of  paths  and  our  inability  to  effectively  enumerate  them  is  what  creates  the  need  to  use  AI  techniques. 

Conventional  programming  languages  provide  effective  constructs  for  developing  algorithms.  They  effectively  handle  sequential 
processing,  iteration,  recursion,  function  calling,  and  data  abstraction.  They  contain  only  a  limited  capacity  to  handle  complex  logic 
definitions.  Smart  systems  must  process  large  amounts  of  logic  to  make  the  decisions  necessary  to  drive  the  algorithmic  processing. 
Logic  that  can  not  be  broken  down  hierarchically  generates  very  complex  software.  Usually  numerous  flags  are  created  and  added  as 
the  problem  is  refined,  making  the  code  more  complex,  incomprehensible,  and  unmaintainable. 

AI  knowledge  based  techniques  were  developed  to  allow  problem  domain  knowledge  to  be  acquired  from  experts  and  encoded  for 
automated  reasoning  and  decision  making.  Knowledge  based  technologies  provide  the  framework  for  real-time  embedded  avionics 
processing.  Knowledge  based  systems  use  special  high  order  language  constructs  to  define  the  logic.  These  constructs  are  called  pro¬ 
ductions  rules.  They  reduce  software  complexity  by  pulling  the  logic  out  of  the  code  and  putting  it  in  the  knowledge  base. 

Pulling  this  complex  logic  out  of  the  cods  reduces  the  complexity  of  the  code  by  eliminating  the  portion  of  the  code  that  represents 
the  complex  logic.  Complex  logic  coded  in  Ada  and  other  conventional  languages,  is  usually  made  up  of  numerous  conditionals,  flags, 
and  switches,  which  are  very  difficult  to  understand  and  maintain.  The  rules  in  the  knowledge  base  are  user  defined  and  are  modular. 
They  are  automatically  linked  to  each  other  and  to  the  algorithmic  code  by  the  inference  engine  during  program  execution.  This  allows 
the  development,  visualization,  and  maintenance  of  the  software  at  a  higher  level  of  abstraction,  and  leaves  the  actual  creation  and 
maintenance  of  the  logical  links  to  the  inference  engine.  See  figure  2. 

Most  AI  systems  do  little  algorithmic  processing,  they  concentrate  on  knowledge  processing.  Embedded  systems  require  hybnd 
conventional  and  AI  solutions,  where  most  of  the  complexity  is  contained  in  the  reasoning  and  decision  making  logic,  while  the  algo¬ 
rithms  contain  the  bulk  of  the  code,  perform  most  of  the  processing  functions,  and  expend  most  of  the  processing  resources 
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•  Separates  the  logic  from  the  Algorithmic  Code. 

•  The  Algorithmic  Code  becomes  less  complex,  more  modular,  and  discrete. 

•  Express  the  logic  at  a  higher  level  of  abstraction  in  the  knowledge  base. 


FIGURE  2:  KNOWLEDGE  BASED  APPROACH 


Effective  smart  avionics  software  must  be  developed  by  combining  features  of  two  separate  technologies:  (1)  conventional  software 
engineering  techniques,  which  provide  powerful  languages,  standards  and  practices;  and  (2)  Artificial  Intelligence  (AI)  techniques, 
which  provide  autonomous  reasonin'-  and  decision  making  capabilities.  These  two  technologies  must  be  effectively  combined  to  pro¬ 
vide  a  “hybrid  software  engineer  ,g  framework”  that  can  support  high  quality,  cost  effective,  and  maintainable  software  for  smart  avi¬ 
onics  systems.  Much  work  has  been  done  each  of  these  areas,  but  little  has  been  accomplished  ir.  efforts  that  combine  all  of  them.  To 
develop  intelligent  real-time  embedded  systems  required  us  lo  pull  together  techniques  from  each  of  these  areas  and  develop  a  method¬ 
ology  that  facilitates  efficient  software  development  and  supports  our  mission  needs. 


AN  ARTIFICIAL  INTELLIGENCE  METHODOLOGY 

The  methodology  selected  to  solve  the  mission  management  problem  uses  distributed,  parallel,  cooperating  expert  systems.  These 
expert  systems  are  functionally  separate  and  communicate  using  message  passing  techniques.  Each  individual  expert  system  requires 
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only  basic  expert  system  capabilities.  All  that  is  needed  is  an  efficient  and  relatively  simple  production  rule  driven  inference  engines 
that  support  english  like  rule  parsing,  interface  to  a  data  base,  and  provide  procedural  and  priority  attachment  facilities. 

Declarative  mission  knowledge  is  contained  in  production  roles.  Standard  high  level  programming  language  functions  ana  proce¬ 
dures  form  the  basis  of  the  lo  /  level  algorithmic  code.  These  procedures  will  be  invoked  and  controlled  by  the  rules,  providing  a  dy¬ 
namic  yet  simple  control  structure. 

The  system  is  data  or  knowledge  driven.  The  inference  engine  invokes  the  appropriate  software  functions  as  determined  by  the  sum. 
of  existing  facts.  Thus,  the  world  state  determines  what  software  is  invoked.  External  systems  and  sensors  directly  assert  facts  and  the 
resultant  system  states  drive  the  inference  process.  Inferencing  ensures  that  appropriate  mission  activities  are  accomplished  at  the  right 
nme.  The  system  continuously  provides  its  best  solutions  to  the  existing  mission  management  problems.  This  AI  methodology  allows 
the  efficient  combination  of  information  from  diverse  sources. 

The  inferencing  techniques  operate  on  sets  of  production  roles.  Production  rules  are  associated  with  individual  system  facts  and  al¬ 
gorithmic  procedures.  Inferencing  for  or  about  a  fact  results  in  the  firing  of  the  rules  associated  with  that  fact.  Facts  are  organized  in  hi¬ 
erarchical,  frame  like,  graph  structures.  Inferencing  techniques  control  heuristic  traversal  of  the  structures.  As  a  result  of  the 
inferencing,  fact  values  are  updated  and  procedures  arc  invoked.  Procedures  can  be  called  directly  using  procedural  attachment  to  the 
rules,  or  indirectly  as  side  effects  of  the  inferencing.  Execution  and  control  of  these  procedures  is  the  primary  objective  of  the  mission 
manager. 

The  system  director  is  implemented  as  an  enhanced  expert  system  that  functions  similar  to  an  AI  blackboard  controller.  The  system 
director  controls  a  global  data  base  or  blackboard,  which  contains  the  mission  plan  and  current  aircraft  and  mission  status.  The  system 
director  controls  the  individual  expert  systems  or  knowledge  sources  directly  by  analyzing  key  blackboard  entries.  It  invokes  specific 
knowledge  sources  to  schedule  events,  analyze  the  blackboard,  or  perform  high  level  mission  functions.  The  knowledge  sources  con¬ 
sist  of  expert  systems,  production  roles  or  procedure  calls  to  system  functions. 

The  primary  communication  medium  between  the  mission  manager  processes  is  the  blackboard  or  global  data  base.  All  communi¬ 
cation  between  the  experts  and  algorithmic  procedures  is  through  this  structure.  There  are  three  major  data  bases  in  the  blackboard. 

1)  The  Aircraft  Status  Data  Base  contains  a  hierarchy  of  facts  that  describe  current  aircraft,  equipment,  and  weapon  status  It  con¬ 
tains  information  on:  switch  positions,  equipment  modes,  fuel  status,  and  aircraft  degradations. 

2)  The  Mission  Status  Data  Base  contains  a  hierarchy  of  facts  that  describe  current  mission  status.  It  contains  information  on  current 
mission  phase,  mission  plan,  mission  priorities,  mission  objectives,  rules  of  engagement,  and  lists  mission  actions  as  complete  or  pend¬ 
ing 

3)  The  Mission  Plan  contains  all  knowledge  required  to  carry  out  the  mission.  It  describes  mission  phases,  priorities,  flight  plan, 
sensor  control  planning,  and  all  scheduled  mission  and  aircrew  tasks,  switch  actions,  and  procedures.  It  serves  as  the  contract  between 
the  aircrew  and  the  automated  mission  manager  Knowledge  of  the  current  mission  plan  provides  the  aircrew  with  .in  understanding  of 
mission  manager  plans  and  actions.  Mission  plan  changes  can  only  be  made  through  the  system  director  and  with  appropriate  aircrew 
coordination. 


AI  ARCHITECTURE  IMPLEMENTATION 

The  architecture  implemented  to  solve  the  real-time  inflight  mission  manager  probiem  uses  only  basic  knowledge  based  techniques 
distributed  into  an  efficient  cooperating  parallel  environment.  This  knowledge  based  software  consists  of  three  major  elements,  data 
base,  rule  base,  and  inference  engine. 

The  Data  Base 

The  data  base  or  fact  base  contains  all  currently  known  facts.  Frets  are  similar  to  programming  language  variables  with  additional 
attribute  that  they  are  implicitly  linked  together  and  to  the  rules  via  the  knowledge  base.  Facts  may  be  of  a  variety  of  types,  such  as:  in¬ 
tegers,  reals,  floats,  enumerations,  or  strings.  Facts  are  declared  by  stating  a  name  and  type.  Multiple  values  for  each  fact  may  exist 
through  multiple  fact  instantiations,  and  the  facts  may  be  grouped  into  composite  structures  (records). 

The  Rule  Base 

The  production  rules  contain  the  logic  and  knowledge  for  the  specific  problem  being  solved.  They  define  specific  aspects  of  the 
problem  and  what  to  do  under  those  circumstances.  They  are  modular  and  discrete  and  provide  a  kind  of  relational  approach  to  logic 
definition.  The  rules  .ontain  premises  and  conclusions.  The  premises  define  the  context  under  which  the  conclusions  should  be  execut¬ 
ed.  The  premises  conviins  facts  and  possibly  constants,  function  calls,  or  algorithmic  expressions.  Rule  conclusions  contain  commands 
to  invoke  rules  or  proicdures  or  update  fact  values.  Facts  updated  in  the  conclusion  may  trigger  more  rules  to  fire. 

The  knowledge  bast  contains  all  of  the  systems  declarative  knowledge.  Face  rnd  rules  are  both  declared  in  the  knowledge  base  file. 
See  figure  3. 

The  Inference  Engine 

The  inference  engine  is  the  software  that  drives  the  automated  reasoning  process.  The  inference  engine  parses  the  knowledge  base, 
links  the  facts  to  the  rules,  and  perf  orms  automated  ron-time  inferencing.  it  is  the  key  element  in  the  automated  reasoning  process.  The 
inference  engine  must  be  efficient  and  shoutd  impose  as  little  overhead  as  possible. 

The  knowledge  base  parser  reads  the  fact  definitions  and  interprets  production  rules.  It  creates  efficient  data  structures  that  allow  fast 
access  to  the  rules  and  facts.  The  parser  links  the  rules,  facts,  and  procedures  together  by  creating  a  list  of  rules  for  each  fact.  A  fact’s 
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FACTS 

counter_nid>i 

currer.ljada' 

rad«r_mode 

<ircnlt.albhKi: 

nummum.iliitude 


cnumenued_f»ct  (true,  false) 
enumeraterLfact  (gun,  SAM) 
enumerated_f»ct  (tracking,  locked_on,  firing) 
real_fact 
real.faa 


RULES 

Rule.l 

IF  ci'trent.ndar  is  SAM  k  radar.mode  is  tracking 

THEN  ASSERT  coumer_radar 

Rule.2 

IF  counter_radar  &  autrafi_altitude  <=  minimum  jUutude 

THL  '  UPDATE  radar.countermeasure  TO  jam 

Rulc.3 

IF  counier.radar  A  airciafL.altitude  >  rmnimum_altitude 

TIEN  UPDATE  aircraft_alotude  TO  mimmum_alnUKie 


•  A  sample  knowledge  base  illustrating  chaining.  When  RuleJ  is  fired,  the  fact  counter  radar  is  updated.  This  triggers 
the  premises  of  Rule_2  and  RuleJ  to  be  tested.  If  the  result  of  testing  c.  rule's  premise  is  true,  then  its  conclusion  is 
fired,  causing  other  facts  to  be  updated,  and  in  turn  other  rule  premises  to  be  tested,  and  so  on. 


FIGURE  3:  SAMPLE  KNOWLEDGE  BASE 


rule  list  is  constructed  by  examining  the  premises  ot  each  rule.  If  a  rule’s  premise  relerences  a  fact,  then  that  rule  is  placed  on  the  tact’s 
rule  list. 

The  inference  engine  executes  an  iteratively  recursive  inferencing  process.  It  performs  rule  tesung  and  firing,  and  fact  value  updat¬ 
ing. 

When  a  fact  value  is  updated  all  of  the  rules  on  its  fact  list  are  iteratively  tested.  Rule  testing  is  the  process  of  determining  if  the 
premise  is  true.  It  involves  obtaining  fact  values,  and  resolving  the  premise  expressions.  If  the  premise  is  true  then  the  rule  is  fired 

Rule  firing  is  the  process  of  executing  the  rule’s  conclusion.  If  the  process  of  executing  a  conclusion  statement  updates  another  fact  val¬ 
ue,  then  the  process  is  interrupted.  The  inference  engine  recurses  and  starts  the  process  over  with  the  new  fact,  hence  the  designation 
“iterative  recursion.” 


Code  Generator 

The  knowledge  base  code  generator  encodes  the  interpreted  rules  and  facts  into  procedural  code.  The  resultant  cede  allows  the  inter¬ 
preted  aspects  of  the  inference  process  to  be  replaced  by  direct  compiled  code  and  access.  This  speeds  up  the  inferencing  process  by 
several  orders  of  magnitude.  Each  rule  is  turned  into  a  procedure  and  the  rule  fact  links  are  turned  into  an  efficient  lookup  table. 

Knowledge  Driven  Programming 

The  assertion  of  a  new  fact  value  automatically  triggers  the  appropriate  rules  to  be  invoked.  This  is  termed  knowledge  driven  pro¬ 
gramming.  Rule  execution  generates  new  facts,  which  cause  more  rules  to  be  executed.  This  iterauve  process  is  forward  chaining.  See 
Figure  4. 

In  knowledge  driven  systems  the  assertion  of  facts  automatically  initiates  the  appropriate  processes.  External  sensors  and  conditions 
can  automatically  focus  and  control  system  actions  by  their  collective  stales.  This  provides  an  extremely  efficient  processing  paradigm. 

Knowledge  driven  programming  is  efficient  because  the  logic  in  the  knowledge  base  can  be  interpreted  by  the  user  and  the  computer. 
Changes  in  system  states  are  noted  by  new  values  in  the  data  base.  Key  states  automatically  trigger  the  appropriate  code  to  be  executed. 

System  Functions  and  Procedures 

Standard  programming  functions  and  procedures  make  up  the  bulk  of  the  programming  code.  These  are  modular  and  discrete,  and 
are  invoked  and  controlled  by  the  rules.  Their  interactions  are  defined  in  the  knowledge  base.  This  provides  a  dynamic  yet  simple  con¬ 
trol  structure  that  requires  little  control  code  to  be  developed.  Very  little  time  is  spent  developing  complex  control  code,  as  the  control 
is  maintained  in  the  knowledge  base.  These  procedures  perform  a  variety  of  tasks,  but  perform  little  reasoning  and  decision  making. 

Debugging  Facilities 

Smart  embedded  avionics  systems  are  usually  highly  automated  and  do  need  little  operator  interface.  However,  debugging,  testing, 
validation,  and  development  requires  an  interactive  environment.  The  interactive  environment  allows  tact  and  rule  inspection  and  logic 
tracing. 

The  operator  is  able  to  inspect  or  change  fact  values  which  allows  detailed  analysis  of  system  states  and  provides  user  controlled 
state  transitions.  It  is  essential  that  on-line  logic  traces  are  available.  Logic  tracing  is  a  simple  process  because  the  inference  engine  is 
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•  Fact  Base  contains  state  information  of  the  system. 

•  Rule  Base  contains  knowledge  (or  production  rules)  stored  as  If-Then  statements  defining  fact  relationships. 

•  Forward  Chaining  is  the  iterative  process  of  inferencing  using  production  rules. 

•  Inference  Engine  combines  known  facts  and  relationships  to  crea’c  new  fact  values  and  control  program  execution 

FIGURE  4:  KNOWLEDGE  BASE  INFERENCE  METHODOLOGY 

the  focus  of  all  logic  related  activity.  All  facts  are  updated,  and  all  rules  are  tested  and  fired  by  it.  Tracing  rule  accesses  and  fact  value 
updates  are  two  essential  areas  of  logic  tracing. 

Real-Time  Performance 

Real-time  performance  can  be  achieved  by  talcing  advantage  of  the  constraints  and  restrictions  inherent  in  embedded  avionics  sys¬ 
tems.  These  include:  limited  operator  interface,  well  defined  system  interfaces,  simple  data  structures,  well  established  requirements, 
and  rigorously  defined  algorithmic  software  procedures.  They  provide  bounds  for  the  A1  problem  by  limiting  the  scope  of  the  tun-time 
problem.  It  eliminates  the  need  for  symbolic  processing,  which  requires  costly  run-time  interpretation,  examination,  and  decomposi¬ 
tion.  All  variables  or  facts,  which  are  not  numeric,  can  be  statically  enumerated. 

Other  keys  to  real-time  performance  are:  reducing  overhead,  providing  good  procedural  attachment  facilities,  and  increasing  proces¬ 
sor  throughput. 

The  iteratively  recursive  paradigm  is  well  suited  for  real-  time  performance.  It  allows  the  development  of  an  inference  engine  that 'S 
very  compact,  and  consumes  little  processing  overhead,  since  it  contains  only  several  procedures  and  small  data  structures  for  each  fact 
and  rule.  It  is  only  invoked  when  facts  are  updated  that  may  trigger  rules  to  be  fired. 

There  is  no  run-time  overhead  in  determining  which  rules  are  candidates  for  firing  or  the  order  of  firing.  They  are  fired  sequentially 
from  the  fact’s  rule  list.  Also,  since  the  problem  is  decomposable  by  facts  and  rules  it  can  be  broken  down  and  solved  concurrently. 

Cooperating  Expert  Systems 

Concurrent  distributed  processing  can  be  accomplished  by  breaking  the  knowledge  driven  system  up  into  multiple  cooperaung  ex 
pert  systems.  Each  expert  system  operates  on  its  own  set  of  facts  and  rules.  Cooperation  and  communication  are  accomplished  via 
shared  facts. 

Defining  expert  systems  interfaces  is  a  very  straight  forward  process.  Each  expert  system  has  its  own  knowledge  base,  shared  fact-, 
are  declared  in  the  tact  definitions.  When  the  value  of  a  shared  fact  is  changed  by  an  expert  system,  a  message  is  sent  to  the  other  expert 
systems  with  the  new  value. 

When  another  expert  system  changes  the  value  of  a  shared  fact,  a  message  is  received  by  the  expert  stating  the  fact  name  and  the  new 
value.  This  message  is  stored  in  a  queue  and  is  not  processed  until  the  system  finishes  the  current  recursive  processing.  The  messages 
are  then  processed  in  turn.  The  messages  change  fact  values  and  may  cause  inferencing. 

Messages  from  other  experts  are  treated  exactly  like  inputs  from  sensors,  systems,  or  the  operator.  There  is  very  little  processing  or 
size  overhead  in  this  method  of  distributed  processing. 

Ada  REAL-TIME  INFERENCE  ENGINE  (ARTIE) 

To  develop  an  mission  manager  to  run  on  a  parallel  embedded  environment  and  provide  all  of  the  required  capabilities  it  was  neces¬ 
sary  to  develop  a  real-time  inference  engine.  Several  generations  have  been  developed,  in  various  programming  languages,  using  a  va¬ 
riety  of  knowledge  cased  techniques  and  features.  The  current  inference  engine  includes  all  of  the  features  outlined  above.  It  is  written 
in  Ada  and  is  called  the  Ada  Real-Time  Inference  Engine  or  ARTIE.  ARTIE  is  very  fast,  compact,  and  runs  interactively  or  embedded, 
alone  or  with  multiple  cooperating  expert  systems 

ARTIE  consists  of  approximately  13,000  lines  of  Ada  source  code,  and  performs  the  following  key  functions,  knowledge  base  pars¬ 
ing,  rule-fact-code  linking,  on-ltne  trace  and  debug  facilities,  on-line  rule  and  fact  modification,  and  runtime  inferencing.  ARTIE  fires 
over  1400  rules  per  second  on  a  typical  work  station  in  the  interpretive  mode,  it  is  significantly  faster  using  the  compiled  code  genera- 
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tor.  ARTIE  has  been  successfully  executed  without  modification  on:  Apollo  3000, 4000,  and  590,  Sun  2/60,3/60  and  4/60,  Vax,  Mi- 
ctovax,  and  Silicon  Graphics  IRIS  workstations.  ARTIE  also  r  tns  on  a  parallel  processor  which  consists  of  10  embedded  68030 
processors  on  a  VME  backplane. 

ARTIE  can  function  as  a  stand-alone  system  controlled  by  the  operator  through  an  interactive  interface,  or  autonomously  embedded 
in  an  Ada  program  controlled  by  external  Ada  code  that  is  knowledge  driven. 


DEVELOPING  INTELLIGENT  SYSTEMS 

Developing  intelligent  systems  requires  the  use  of  specialized  software  development  techniques.  Knowledge  based  software  devel¬ 
opment  techniques  can  reduce  the  time  and  costs  required  to  develop  smart  avionics  software.  In  addition,  it  facilitates  the  creation 
of  comprehensible,  maintainable,  and  verifiable  systems.  Knowledge  based  software  development  is  a  methodology  that  uses  Artifi¬ 
cial  Intelligence  knowledge  based  techniques  to  provide:  1)  less  complex  software;  and  2)  rapid  development,  coding,  test,  and  ver¬ 
ification  of  intelligent  and  autonomous  software. 

Software  complexity  is  reduced  by  extracting  embedded  decision  and  control  software  from  the  complex  code  and  putting  it  in 
a  knowledge  base.  Decision  and  control  software  is  the  reasoning  software  that  represents  the  logic  that  drives  the  system  by  link¬ 
ing  together  the  algorithms  that  perform  system  tasks.  This  software  is  not  represented  efficiently  in  conventional  high  order  languag¬ 
es.  It  will  be  placed  in  the  knowledge  base. 

The  knowledge  ba:' contains  high  level,  user  understandable,  modular  descriptions  of  the  decision  and  control  logic.  These  mod¬ 
ular  logic  descriptions  are  the  production  rules.  Removing  the  logic  from  the  complex  code  leaves  behind  simple,  discrete  algorithms 
and  procedures.  The  knowledge  base  is  interpreted  by  the  inference  engine  which  creates  the  actual  logic  links. 

Reduced  software  complexity  makes  it  possible  to  create  software  that  is  comprehensible  as  well  as  verifiable  and  maintainable. 
The  knowledge  base  provides  a  common  language  and  interface  between  the  customer,  systems  engineers,  coders,  testers,  and  main¬ 
tained.  An  interpreted  inference  engine  with  an  automatic  code  generator  allows  efficient  incremental  or  iterative  software  develop¬ 
ment. 

Rapid  software  development  is  accomplished  by  bringing  the  entire  software  team  together  and  using  the  advanced  features  of  the 
interpreted  inference  engine  to  develop,  test,  and  verify  the  software  The  automatic  code  generator  is  invoked  to  rapidly  create 
the  embedded  high  order  language  code. 

The  key  to  knowledge  based  software  development  is  having  a  inference  engine  that:  is  very  small,  efficient,  embeddable,  per¬ 
forms  real  time  inferencing,  provides  on-  line  reasoning  and  software  execution  trace  and  debug  facilities,  supports  high  level 
code  generation  for  embedded  operations,  and  is  coded  m  a  high  order  language  using  standard  engineering  practices  and  proce¬ 
dures. 

With  an  understanding  of  the  inference  engine,  experts  can  develop  the  rules  to  solve  the  problems.  Computer  programmers  need 
only  assist  the  experts  and  code  the  low  level  functions  described  by  the  experts  as  necessary  to  provide  information  or  processing  to  be 
controlled  by  the  rules. 

The  knowledge  based  software  development  process  is  streamlined  to  provide  two  major  capabilities:  rapid  prototyping  and  efficient 
performance.  To  meet  these  to  objectives  the  development  process  uses  multiple  levels  of  interpretation  and  symbolic  processing. 
Software  tools  automate  the  transitions  between  these  levels. 

In  the  early  stages  of  intelligent  system  development  rapid  prototyping  and  quick  tum  around  are  essential.  This  is  true  because  de¬ 
velopment  of  intelligent  systems  requires  an  iterative  approach.  The  systems  are  complex  and  logic  errors  will  be  found  and  must  be 
eliminated.  The  rapid  prototyping  will  be  accomplished  by  using  an  interpreted  inference  engine  and  placing  as  much  of  the  new  code 
as  possible  in  the  knowledge  bases.  Modular  knowledge  bases  allow  incremental  rule  base  development,  similar  to  Ada's  packages. 
The  interpreted  inference  engine  reduced  the  required  number  of  recompilations  and  provides  on-  line  debugging  and  modification.  See 
figure  5. 


generation,  and  shorter  development  lime. 


FIGURE  5:  RULE  BASED  SOFTWARE  DF.VELOPMEN  r  PROCESS 
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In  later  stages  of  development  real-time  performance  and  compact  size  become  the  priorities.  At  this  stage,  tested  knowledge  bases 
are  compiled  directly  into  procedural  code.  These  rules  are  accessed  directly  like  programming  language  procedures.  Modification  of 
these  rales  becomes  more  time  consuming  and  complex  but  this  is  not  a  key  factor  at  this  stage  of  development. 

Faciliues  are  provided  throughout  the  stages  of  software  development  to  allow  debug  and  logic  tracing,  even  on  the  target  processor. 
Facilities  are  provided  to  allow  the  use  of  multiple  knowledge  bases  which  may  be  at  different  levels  of  development,  some  may  be  in¬ 
terpreted,  while  others  are  already  compiled  as  procedural  code.  The  result  is  a  very  efficient  process  that  allows  rapid  prototyping 
while  preserving  high  performance. 


MISSION  MANAGER  DEVELOPMENTS 

Several  Boeing  research  projects  have  contributed  to  our  real-time  mission  management  knowledge  and  experience.  These  include: 
(1)  On  Board  Mission  Management,  a  project  that  developed  a  prototype  strategic  inflight  mission  manager,  (2)  Advanced  Tactical 
Cockpit,  a  project  to  develop  and  demonstrate  a  tactical  mission  manager  in  a  domed  simulator,  (3)  Knowledge  Based  Software  Devel¬ 
opment,  a  project  that  developed  a  real-time  Ada  based  inference  engine,  knowledge  based  development  techniques,  methods,  practic¬ 
es,  and  tools,  and  (4)  The  Boeing  Advanced  Avionics  Test  Bed,  a  project  that  provides  a  Boeing  aircraft  for  flight  testing  and  evaluating 
advanced  avionics  systems. 

On  Board  Mission  Manager  (IR&D  BMA-03I) 

A  strategic  on  board  mission  management  system  (OBMM)  for  bomber  aircraft  was  developed  at  Boeing.  The  system  provides  in¬ 
flight  mission  replanning,  automated  mission  statusing  and  aircraft  health  monitoring,  and  evaluates  mulu-sortie  nuclear  deconfiictions. 
It  also  models  and  assesses  the  threat  environment  and  controls  on  board  radio  frequency  sensors.  Aircrew  interface  is  via  multiple 
color  displays,  a  tracking  handle,  and  programmable  touch  panels. 

The  OBMM  system  will  aid  the  strategic  aircrew  in  accomplishing  its  mission  under  the  myriad  of  constraints  and  condiuons  that 
exist  in  the  current  and  future  strategic  environment.  It  collects,  synthesizes,  analyzes,  filters,  and  controls  information  gathered  by 
modem  aircraft  sensors  and  avionics,  and  presents  situations  and  status  with  alternative  actions  to  the  aircrew.  03MM  provides  adap¬ 
tive  sensor  control,  mission  planning  and  replanning,  aircrew  interfaces,  and  aircraft  status  and  monitoring  as  dictated  by  the  mission. 

The  on  board  mission  manager  is  controlled  by  distributed,  parallel  cooperating  expert  systems.  OBMM  experts  and  procedures 
communicate  with  each  other  through  the  blackboard,  or  global  data  bases.  Global  data  bases  contain  the  mission  plan  along  with  air¬ 
craft,  weapon  and  mission  status. 


Tactical  Mission  Manager  (IR&D  BMA-897) 

An  advanced  tactical  fighter  cockpit  mission  manager  that  manages  air-to-ground  tactical  fighter  and  unmanned  air  vehicle  missions 
was  developed  at  Boeing.  It  performs  inflight  mission  management,  automated  sensor  control,  threat  recognition  and  countermeasure 
selection,  inflight  mission  replanning,  and  advanced  control  and  display  processing.  The  system  was  developed  in  Ada  and  will  on 
many  platforms.  The  system  was  demonstrated  in  a  distributed  Silicon  Graphics  environment,  linked  into  off-the-shelf  simulation  and 
graphics  support  tools.  The  system  was  flown  man-in-the-loop  and  in  a  fully  autonomous  mode  to  represent  an  unmanned  air  vehicle. 
See  figure  6. 


Simulation  Proeassor(s) 


Cockpit 


?  His  intelligent  simulation  architecture  depicts  major  Tactical  Mission  Manager  components  and  their  interrelation¬ 
ships. 


FIGURE  6:  TACTICAL  MISSION  MANAGER  ARCHITECTURE 
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A  key  capability  of  the  tactical  mission  manager  is  real-  time  threat  avoidance.  The  system  is  designed  to  use  knowledge  of  the  low 
level  mission  environment  to  determine  the  best  threat  countermeasure.  Threat  countermeasures  are  not  limited  to  electronic  jamming 
or  spoofing,  but  may  include:  aircraft  maneuvers,  altitude  changes,  expendable  countermeasures,  or  mission  replanning. 

Mission  planning  and  replanning  includes  weapon  selection,  mode  control,  and  delivery  tacucs  optimizauon.  When  the  mission  ob¬ 
jectives,  threat  environment,  or  aircraft  systems  capabilities  deviate  from  that  required  by  the  planned  mission;  the  mission  manager  au¬ 
tomatically  replans  the  necessary  mission  segments.  The  replanning  may  require  reconfigured  weapons,  a  different  delivery  tacuc,  or 
an  alternate  flight  profile.  This  capability  provides  real-erne  inflight  tactics  and  mission  management. 

Advanced  Avionics  Testbed  (IR&D  BMA  -032) 

Key  aspects  of  these  prototype  projects  have  been  tested  and  flown  on  Boeing’s  Advanced  Avionics  Testbed  Aircraft  (AAT),  a  Boe¬ 
ing  720  aircraft  that  has  been  modified  and  instrumented  to  allow  inflight  evaluation  and  testing  of  advanced  avionics  systems.  The 
AAT  is  dedicated  to  flight  testing  advanced  offensive  and  defensive  systems.  Equipped  with  state-of-the-art  sensors,  instrumentauon, 
and  communications  and  navigation  equipment  the  AAT  has  provided  a  facility  for  real  world  testing  of  inflight  mission  manager  com¬ 
ponents.  Real  world  conditions  include:  threat  avoidance  (via  a  Loral  radar  warning  receiver);  inflight  mission  replanning;  threat  recog¬ 
nition;  simulated  countermeasure  selection;  advanced  controls  and  displays;  sensor  management  (forward  looking  infrared  radar, 
millimeter  wave  radar,  laser  line  scanner,  optical  imagery);  intelligent  navigation  (global  positioning  system,  inertial  navigation  unit, 
digital  map  aided  terrain  following);  advanced  situation  awareness  (multiple  color  displays). 


LESSONS  LEARNED 

Several  lessons  were  learned  in  the  process  of  developing  real-time  inflight  mission  managers  and  advanced  avionics  systems.  Several 
key  points  follow. 

Complex  AI  facilities  are  not  required  for  state-of-the-art  intelligent  embedded  systems.  They  add  too  much  complexity,  are  too  in¬ 
efficient  and  never  seem  to  quite  fit.  Facilities  such  as  evidential  reasoning,  inheritance,  and  frame  based  a  architectures  were  explored. 
These  are  useful  features,  but  make  the  systems  too  complex  and  slow  for  most  real-time  systems. 

Later  versions  of  the  real-time  inference  engine  are  as  streamlined  as  possible  A  key  requirement  that  evolved  was  to  provide  only 
the  necessary  AI  facilities  implemented  with  the  least  overhead  possible. 

Intelligent  embedded  systems  can  achieve  real-time  performance  by  using  knowledge  based  AI  techniques.  They  can  perform  the 
very  intelligent  and  automated  tasking  required  by  modem  combat  systems.  These  systems  can  achieve  very  high  performance  levels 
using  only  basic  knowledge  based  concepts.  The  inference  engine  must  be  extremely  efficient. 


By  using  knowledge  based  techniques  the  program  logic  is  pulled  out  of  the  code  and  put  in  a  knowledge  base,  where  it  is  under¬ 
standable,  modular,  and  modifiable.  This  greatly  reduces  the  complexity  of  logic  intensive  software  and  increases  software  quality 

The  inference  engine  provides  an  interactive  user  interface  for  debugging.  Due  to  the  complexity  of  the  logic  and  the  number  of 
paths  through  the  system,  it  is  essential  that  the  operator  have  complete  visibility  to  system  performance  during  all  stages  of  system  de¬ 
velopment  The  inference  engine  also  provides  interactive  debugging  in  the  final  embedded  avionics  environment. 
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ABSTRACT 


A  common  problem  in  the  design  of  expert  systems  is  the  definition  of  rules  from  data  obtained  in  system  operation  or 
simulation.  While  it  is  relatively  easy  to  collect  data  and  to  log  the  comments  of  human  operators  engaged  in  experiments, 
generalizing  such  information  to  a  set  of  rules  has  not  previously  been  a  straightforward  task.  If  an  expert  human  operator  is 
involved,  he  or  she  can  be  interviewed,  in  the  hopes  of  identifying  experiential  knowledge  concerning  what  does  and  does  not  work. 
However,  this  approach  often  is  unsatisfactory,  either  because  the  expert  has  difficulty  verbalizing  techniques  for  performing  the 
mission  or  because  the  mission  is  largely  governed  by  quanutative  issues  not  well  analyzed  by  human  operators.  This  paper  presents 
a  statistical  method  for  generating  rule  bases  from  numerical  data,  motivated  by  an  example  based  on  aircraft  navigauon  with  multiple 
sensors.  The  specific  objective  is  to  design  an  expert  system  that  selects  a  satisfactory  suite  of  measurements  from  a  dissimilar, 
redundant  set,  given  an  arbitrary  navigation  geometry  and  possible  sensor  failures. 

This  paper  describes  the  systematic  development  of  a  Navigation  Sensor  Management  (NSM)  Expert  System  from  Kalman  Filter 
covariance  data.  The  development  method  invokes  two  statistical  techniques:  Analysis-of-Variancc  (ANOVA)  and  the  IDi  algorithm. 
The  ANOVA  technique  indicates  whether  variations  of  problem  parameters  give  statistically  different  covariance  results,  and  the  1D3 
algorithm  identifies  the  relationships  between  the  problem  parameters  using  probabilistic  knowledge  extracted  from  a  simulation 
example  set.  ANOVA  results  show  that  statistically  different  position  accuracies  are  obtained  when  different  navigation  aids  are  used, 
the  number  of  navigation  aids  is  changed,  the  trajectory  is  varied,  or  the  performance  history  is  altered.  By  indicating  that  these  four 
factors  significantly  affect  the  decision  metric,  an  appropriate  parameter  framework  was  designed,  and  a  simulation  example  base  was 
created.  The  example  bas>' contained  over  900  training  examples  from  nearly  300  simulations.  TheID3  algorithm  then  was  applied 
to  the  example  base,  yielding  classification  “rules"  in  the  form  of  decision  trees.  The  NSM  expert  system  consists  of  17  decision  trees 
that  predict  the  performance  of  a  specified  integrated  navigation  sensor  configuration.  The  performance  of  these  decision  trees  was 
assessed  on  two  arbitrary  trajectories,  and  the  performance  results  are  presented  using  a  predictive  metric.  The  test  trajectories  used  to 
evaluate  the  system's  performance  show  that  the  NSM  Expert  adapts  to  new  situations  and  provides  reasonable  estimates  of  sensor 
configuration  performance.  The  paper  shows  how  the  NSM  Expert  finds  the  best  sensor  navigation  strategy  from  a  pool  of  available 
sensors  to  provide  the  highest  position  accuracy. 


NOMENCLATURE 

ej  Total  number  of  examples  with  the  i1*1  attribute  value 

ej  Total  number  of  examples  in  training  set 

F  Computed  F-ratio 

Ha  Alternate  hypothesis 

Ho  Null  hypothesis 

(m)  m1*1  observation  of  the  result 

p(cj)  Probability  of  occurrence  of  the  j*  result  value 

Pij  Probability  of  occurrence  of  j*  result  with  i<h  attribute  value 

Yj,  Sum  of  observed  results  using  the  ilh  value 

Yy  Observed  result  obtained  when  i111  and  fh  values  of  Factors 

_  A  and  B  are  used 

Y i.  Mean  value  of  observed  results  using  the  i* 
value  of  the  factor  under  investigation 
a,  Effect  of  i1*1  value  of  Factor  A  on  observed  result  Yy 

pj  Effect  ofi*  value  of  Factor  Bon  observed  result  Yy 

(ap)y  Effect  of  interactions  between  i|h  value  of  Factor  A 
and  j1*1  value  of  Factor  B  on  the  observed  result  Yy 
p  Geodetic  distance 

0  Bearing  measurement 

ft..  Mean  of  all  observed  results  in  data  set  being 
used  in  ANOVA  experiment 
5  Confidence  level 
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INTRODUCTION 


Knowledge  acquisition  is  a  major  problem  in  the  development  of  rule-based  systems.  The  tools  developed  to  date  are  not 
designed  to  extract  information  from  data  for  which  no  generalizations  are  known  a  prion.  Instead,  these  tools  either  rely  on  the 
expert  to  provide  examples  from  which  rules  are  generated  or  try  to  capture  the  expert's  problem-solving  methodology  with 
interviewing  techniques  [1].  Unfortunately,  it  is  often  difficult  for  experts  to  describe  their  problem-solving  methods  or  to  detail  the 
factors  that  come  into  play  during  the  resolution  of  a  problem.  It  is  exactly  this  type  of  knowledge  that  is  needed  to  design  rule-based 
systems. 

Since  the  early  1970's,  adaptive  navigation  has  been  viewed  as  a  highly  desirable  candidate  for  development  in  next-generation 
aircraft  [2].  It  is  envisioned  that  future  aircraft  will  have  multi-sensor  capability  for  navigation  tasks  requiring  high  reliability,  optimal 
performance,  and  increased  automauon.  With  multi-sensor  capability,  the  task  of  sensor  configuration  selection  and  management 
will  become  an  additional  pilot  burden. 

The  performance  of  multi-sensor  navigation  systems  (more  commonly  known  as  ''integrated"  or  "hybrid"  systems)  has  been 
explored  since  the  late  1960's  when  results  from  modem  control  theory  provided  techniques  for  sensor  mixing  and  optimal  state 
estimauon  (3).  Hybrid  systems  refer  to  externally  referenced  navigation  systems  that  "aid"  an  on-board  inertial  navigation  system 
(INS)  using  an  optimal  state  estimation  mechanization.  Hybrid  systems  combine  the  high-  and  low-frequency  accuracy  properties  of 
INSs  and  external  navigation  aids  (navaids)  respectively.  Many  radio  navigation  and  on-board  systems  aiding  INSs  have  been 
modelled  and  their  performance  covariance  results  obtained  [4-8].  When  radio  navigation  systems  are  only  partially  operational, 
results  show  that  navigation  performance  is  superior  to  that  of  the  pure  INS  [4],  Therefore  it  becomes  advantageous  to  keep  partially 
operational  systems  as  candidates  for  integrated  sensor  mixing  purposes. 

With  a  large  number  of  available  navaids,  choosing  an  optimal  or  near-optimal  sensor  set  becomes  a  large  combinatorial  problem. 
Convergence  towards  an  optimal  sensor  configuration  requires  an  exhaustive  computer  search  utilizing  simulation  results  as  the  basis 
for  selection.  In  contrast,  a  small  number  of  available  navaids  reduces  the  decision  space  considerably.  Hence,  a  dilemma  occurs; 
increasing  sensor  capability  (and  thus  reliability  and  performance)  increases  decision-making  complexity. 

The  selection  of  an  optimal  configuration  requires  the  application  of  some  decision  criteria.  Most  often,  designers  choose 
between  navaids  based  or.  the  relative  accuracies  of  each  system  using  a  hierarchical  approach  [9].  This  approach  is  "knowledge- 
based"  in  the  sense  that  the  nominal  performance  of  each  navaid  is  well-known  and  that  this  knowledge  is  built  into  the  sensor 
hierarchy.  The  current  hierarchical  designs  are  not  as  "robust"  with  respect  to  sensor  availability  and  performance  changes  as  is 
necessary  for  future  sensor  management  systems  [10]  Instead,  these  hierarchies  represent  "rules-of-thumb"  that  are  useful  in  only 
the  simplest  cases.  They  do  not  resolve  sensor  configuration  problems  when  more  detailed  information  must  be  considered  -  for 
example  when  the  number  of  each  available  navaid  is  specified,  when  partially  operational  systems  remain  viable  candidates,  and 
when  trajectory  effects  degrade  system  performance.  It  becomes  necessary  to  explore  factors  other  than  the  performance  of 
nominally  operating  navaids  to  determine  how  these  factors  affect  the  decision-making  process,  and  to  fully  exploit  the  potential  of 
hybrid  systems. 

The  Analysis-of- Variance  statistical  technique  (ANOVA)  [1 1]  was  used  to  identify  the  factors  that  cause  variation  in  navigation 
performance.  Once  the  important  factors  were  identified,  the  relationships  between  them  were  determined.  The  ID3  algorithm 
[12,13],  an  inductive  inference  technique  based  on  the  probabilistic  occurrence  of  events,  was  used  to  find  these  attribute 
rclauonships. 

The  development  of  a  navigation  sensor  management  expert  system  using  the  ANOVA/ID3  technique  [14,15]  is  described  in  this 
paper.  The  NSM  system  controls  the  selection  of  multi-sensor  configurations.  The  methodology  is  applicable  to  any  problem  where 
the  development  of  knowledge  bases  from  multi-factor  data  studies  is  desired. 

INTEGRATED  NAVIGATION  SYSTEMS 

Optimal  estimation  techniques  are  used  to  combine  inertial  and  radio  navigational  systems  in  order  to  provide  stable  continuous 
inertial  navigation  information  [16].  The  errors  exhibited  by  these  "hybrid”  systems  depend  on  the  accuracy  of  the  aiding  system, 
and  navaid  accuracies  are  functions  of  many  factors,  such  as  navaid  type,  number  of  similar  navaids,  distance  from  the  navaid,  and 
whether  the  aircraft  is  approaching  or  receding  from  the  station.  The  sensor  selection  critena  depend  on  the  relative  importance  of 
these  factors.  Five  external  radio  navigation  and  two  on-board  navaids  were  used  to  update  a  medium-accuracy  (10  Nauncal  MiThr) 
INS.  Hybrid  system  performance  was  simulated  using  the  linearized  inertial  navigation  error  model  and  navaid  measurement  models 
as  inputs  into  the  optimal  estimation  filter.  The  hybrid  errors  were  updated  at  a  specified  navaid  fix  rate.  The  systems  simulated  were 
(1)  Global  Positioning  System  (GPS),  (2)  Long-Range  Navigation  System  (LORAN),  (3)  Tactical  Navigation  System  (TACAN), 
(4)  Distance  Measuring  Equipment  (DME),  (5)  VHF  Omnidirectional  Range  (VOR),  (6)  Doppler  radar,  and  (7)  Air  Data  sensor. 
The  operauonal  theory  and  the  mathematical  models  used  to  simulate  the  navaids  and  the  inertial  navigation  error  model  are  discussed 
in  detail  elsewhere  [14]. 

The  numerically-stable  discrete-time  U-D  implementation  of  the  Kalman  Filter  equations  [17]  was  used  to  mix  the  inertial  system 
and  navaid  information  optimally,  providing  covariance  estimates  of  the  navigation  errors  (e.g.,  north/east  position)  [14,18].  Each 
nonlinear  measurement  equation  was  linearized  with  respect  to  the  inertial  navigation  states  to  obtain  the  observauon  matrix  used  in 
the  U-D  measurement  update  equations.  Since  sensor  errors  were  taken  into  consideration  in  the  measurement  models,  the  inertial 
enor  state  vector  was  augmented  with  the  sensor  shaping  filter  dynamics  (e.g.,  random  bias,  first-order  Markov  model)  to  formulate 
the  hybrid  navigation  model.  Additionally,  the  measurement  noise  time  history  was  simulated.  As  the  aircraft  moves  along  its 
trajectory  relative  to  ground-based  navaid  stations,  the  measurement  noise  characteristics  change.  Therefore  an  equation  for  a 
distance-  or  time-varying  measurement  covariance  matrix  was  found  in  order  to  realistically  model  ground-based  radio  navigation 
systems.  According  to  Ref.  18,  GPS  measurement  noise  increases  in  a  similar  way;  as  the  satellite  descends  near  the  aircraft's 
horizon,  the  noise  increases.  To  simulate  time-varying  measurement  noise  for  the  ground-  and  satellite-based  navigation  systems, 
each  noise  variance  was  modelled  as  the  sum  of  initial  and  range-dependent  variances.  The  latter  component  increases  linearly  with 
the  square  of  the  distance  from  the  station  or  satellite. 

Position  accuracy  was  selected  for  the  rule-based  system  decision  metric.  Here,  position  accuracy  is  defined  as  the  root  sum  of 
squares  (RSS)  of  the  north  and  east  component  errors.  The  RSS  decision  metric  provides  sufficiently  consistent  quantities  to 
compare  hybrid  performances.  For  a  detailed  discussion  of  the  RSS  decision  metric,  the  reader  is  directed  to  Ref.  14. 


HYBRID  NAVIGATION  SIMULATION  RESULTS 


Using  the  RSS  position  error  metric  to  measure  hybrid  system  performance,  the  following  U-D  filter  simulations  were 
performed: 

1 .  Single-type  hybrids:  GPS,  LORAN,  TACAN,  DME,  VOR,  Doppler  Radar,  or 
Air  Data  sensor  aiding  an  INS 

2.  Number  of  stations  used  in  a  single-type  hybrid 

3.  Multi-type  hybrids:  Combinations  of  different  navaid  types  aiding  an  INS 

4.  Aircraft  trajectories  simulated:  High-performance,  commercial,  general  aviation 

Comparisons  of  Single-Type  Hybrid  Performance 

Consider  the  four  ground  stations  A,  B,  C,  and  D  spatially  oriented  with  respect  to  the  high-performance,  commercial,  and 
general  aviation  trajectories  in  Fig.  1.  The  four  ground  stations  are  simulated  as  LORAN  slaves,  TACAN,  DME,  or  VOR  stations. 
Figure  2  shows  the  performance  differences  of  ground-based,  GPS,  and  on-board  type  hybrid  systems.  When  the  results  from  all 
ground  station  A  types  (LORAN,  TACAN,  DME,  VOR)  are  compared  on  the  high-performance  trajectory,  the  relative  performance 
from  best  to  worst  may  be  listed  as  follows:  (1)  LORAN,  (2)  TACAN,  (3)  DME,  and  (4)  VOR.  For  example,  a  hybrid  system 
utilizing  LORAN  Slave  Station  A  provides  better  performance  than  a  hybrid  system  utilizing  TACAN  A;  a  TACAN  A  hybnd  in  turn 
outperforms  a  DME  A  hybrid  which  in  turn  outperforms  a  VOR  A  hybrid.  This  pattern  is  repeated  for  stations  B,  C,  and  D  114). 
The  best  hybrid  performance  was  obtained  from  three  GPS  satellites  aiding  the  INS.  Figure  2  also  shows  how  the  performances  of 
the  Doppler  radar  hybrid  and  the  air  data  sensor  hybrid  compare  with  the  GPS  and  ground-based  navaid  hybrids. 

Referring  to  the  LORAN  results  in  Fig.  3,  there  is  a  striking  variation  in  the  performance  of  the  individual  Stations  A-D;  this 
figure  reveals  that  single  stations  of  the  same  type  aiding  an  INS  give  highly  variable  performance  results.  The  same  variability  in 
performance  of  the  remaining  ground-based  single-stanon  navaids  was  found  [14).  From  Fig.  3,  the  variation  in  Station  A-D's 
performances  is  attributed  to  die  position  of  each  ground  station  relative  to  the  aircraft's  trajectory.  For  example,  LORAN  Slave  A 
gives  the  smallest  position  error  of  the  four  stations;  referring  to  Fig.  1,  the  aircraft  makes  a  close  approach  to  Slave  A  on  the 
trajectory's  second  leg.  Hence  the  RSS  error  becomes  very  small.  These  errors  begin  to  increase  towards  the  end  of  the  trajectory 
leg,  due  to  the  increasingly  uncertain  north  component.  In  contrast,  LORAN  Slaves  B,  C,  and  D  are  farther  from  the  aircraft's 
trajectory.  The  first  trajectory  leg  results  in  good  relative  north  information  to  B,  C,  and  D,  whereas  the  east  component  uncertainty 
grows  due  to  the  lack  of  relative  east  information.  The  variations  in  performance  observed  from  Stations  A-D  are  due  to  trajectory 
effects;  using  Station  B  instead  of  A  to  update  the  INS  is  equivalent  to  using  A  and  changing  the  aircraft's  trajectory. 

Effect  of  Increasing  the  Number  of  Navaids  in  a  Hybrid  System 

Next,  the  effect  of  the  number  of  ground  stations  was  studied  by  simulating  all  possible  combinations  of  single-,  double-,  and 
tnple-station  hybrids  formed  from  Stations  A-D.  There  are  six  possible  combinations  of  two  stations  and  four  combinations  of  three 
stations  uiat  may  be  integrated  to  aid  the  INS.  These  simulations  were  earned  out  for  LORAN,  TACAN,  DME  and  VOR. 


Figure  I.  Aircraft  Trajectories  Used  in  Simulations 


Referring  to  the  LORAN  results  in  Fig.  4,  the  performance  variation  among  the  double-station  combinations  and  triple  station 
combinations  is  less  pronounced  than  the  single-stauon  variations.  The  magnitude  of  the  RSS  errors  decreases  dramatically  when 
two  stations  are  used  instead  of  one  station.  The  RSS  errors  decrease  further  when  three  stations  are  used,  although  the  magnitude 
differences  are  not  as  great.  The  RSS  magnitudes  of  the  double-  and  triple-station  combinations  are  much  lower  because  the  aircraft 
receives  more  complete  navigation  information  as  the  number  of  stations  increases.  This  also  explains  why  there  is  much  more 
variation  in  the  results  for  the  double  station  combinations  than  for  the  triple  stations.  Similar  performance  trends  were  observed  for 
GPS,  TACAN,  DME,  and  VOR  [14). 

Effect  of  Trajectory  on  Hybrid  Performance 

It  already  has  been  shown  that  an  aircraft's  trajectory  relative  to  a  single  ground  station  hybrid  plays  an  important  role  in  the 
estimator's  performance.  The  RSS  results  in  Fig.  5  illustrate  the  performance  differences  of  the  LORAN  Slave  A  hybrid  on  the  high- 
performance,  commercial  transport,  and  general  aviation  trajectories.  Two  parameters  that  contribute  to  these  performance 
differences  are  distance  to  a  station  and  heading  with  respect  to  a  station.  A  third  trajectory  parameter  that  contributes  to  a  hybrid 
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Figure  2.  Performance  of  Satellite,  Ground 
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Figure  3.  Performance  of  Single-Station  LORAN 
Navaidc  Aiding  an  INS 
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Figure  4.  Performance  of  Double-Station  LORAN 
Navaids  Aiding  an  INS 


system's  performance  is  the  number  of  heading  changes  along  the  trajectory.  The  effect  of  heading  changes  is  discussed  in  more 
detail  in  [14].  Trajectory  factors  affect  the  INS  dynamics,  which  in  turn  affects  the  error  estimation  performance.  The  trajectory 
factors  also  change  the  measurement  dynamics  since  the  measurements  depend  on  the  trajectory's  geometric  properties  and  aircraft 
states  (such  as  velocity).  The  results  in  Fig.  5  clearly  show  that  when  the  trajectory  changes,  the  navaid  selection  decision  most 
likely  changes  as  well,  since  the  relative  accuracies  of  the  navaids  change. 


Hybrid  Performance  of  Mixed  Navaids 


Figure  6  shows  various  combinations  of  integrated  navaids.  The  individual  performances  of  LORAN  Slave  B,  Doppler  radar, 
and  Air  Data  hybrids  are  shown  in  Fig.  6  along  the  high-performance  trajectory.  The  LORAN/Doppler  and  LOR  AN/A'r  Data  hybrids 
also  are  shown  for  comparison.  Both  combinations  gave  better  results  than  their  individual  components  operating  alone.  For 
example,  the  LORAN/Doppler  combination  outperformed  the  LORAN  hybrid  and  the  Doppler  hybrid;  similarly,  the  LORAN/Air 
Data  combination  performed  better  than  either  the  LORAN  or  the  Air  Data  sensor  hybrids  alone.  The  latter  combination  did  slightly 
better  than  Doppler  hybrid  on  this  trajectory  after  the  initial  transient  period.  These  results  show  that  good  navigation  performance  is 
still  possible  when  a  "failed"  LORAN  system  (only  one  slave  station  operational)  is  integrated  with  an  on-board  navaid  such  as 
Doppler  radar  or  a  standard  equipment  Air  Data  sensor. 

Effect  of  Navigation  Sensor  Reconfiguration  on  Hybrid  Performance 

This  section  shows  how  reconfiguration  can  be  used  to  improve  hybnd  filter  performance,  resulting  in  an  optimal  navigation 
strategy.  Here,  the  term  "reconfiguration"  is  exemplified  as  follows:  If  navaids  "A"  and  "B"  give  the  best  filter  performances  on 
trajectory  legs  “1"  and  ”2"  respectively,  then  the  filter  can  be  reconfigured  (changed)  to  use  navaid  B  rather  than  A  on  the  second 
trajectory  leg.  The  results  in  the  previous  sections  demonstrated  that  the  integration  of  several  systems  gives  greatly  improved 
performance  over  the  use  of  each  system  by  itself.  However  a  dilemma  arises  if  computational  resources  on  board  the  aircraft  prevent 
the  filter  from  using  more  than  one  ground  station  for  real-time  measurement  processing.  This  situation  can  occur  if  pan  of  a 
distributed  computer  architecture  or  parallel  processing  system  fails.  A  navigation  expert  system  should  have  the  capability  of 
recommending  a  navigation  strategy  in  this  situation  in  addition  to  making  recommendations  when  the  navigation  computational 
resources  are  fully  operational. 

Figure  7  shows  the  performance  obtained  when  the  best  TACAN  station  information  is  used  on  each  leg  of  the  high-performance 
trajectory  of  Fig.  1.  In  the  figure,  TACAN  station  A  provides  more  accurate  position  information  than  TACAN  station  C  on  the  first 
two  trajectory  legs  (from  0  to  35  minutes).  On  the  last  two  legs.  TACAN  station  C  provides  better  information  to  the  filter  than 
TACAN  station  A.  When  TACAN  station  A  is  reconfigured  to  TACAN  station  C,  the  reconfigured  filter  provides  better  position 
estimates  than  does  a  TACAN  C  filter  operating  on  the  last  two  trajectory  legs.  This  is  because  the  good  measurement  information 
from  TACAN  A  is  propagated  in  time. 

DEVELOPMENT  OF  A  NAVIGATION  SENSOR  MANAGEMENT  EXPERT  SYSTEM 


This  section  describes  a  novel  methodology  that  uses  established  statistical  techniques  to  develop  the  NSM  expert  from  the 
simulation  daf.  The  primary  function  of  this  expert  system  is  to  select  the  external  navaid  sensors  that  provide  the  smallest  possible 
RSS  position  error  from  a  large  set  of  available  sensors.  The  Analysis  of  Variance  (ANOVA)  technique  [1 1)  is  used  to  identify  the 
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Figure  5.  Comparison  or  RSS  Results  for  LORAN 
Hybrids  on  Three  Different  Trajectories 


Figure  6.  Comparison  of  RSS  Results  with  LORAN,  Timc.rn.Ma 
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factors  that  make  statistically  significant  contributions  to  the  decision  metric.  Then,  the  ID3  algorithm  determines  the  relationships 
between  these  factors  [11,13],  The  mathematical  formulations  of  the  ANOVA  and  ID3  techniques  are  briefly  reviewed,  followed  by 
a  discussion  on  how  these  techniques  were  applied  to  the  covariance  data. 

Mathematical  Formulation  of  ANOVA 

Consider  that  two  factors,  A  and  B,  are  to  be  studied  concurrently  m  an  ANOVA  "experiment".  The  ANOVA  experiment 
identifies  the  two  factors'  effects  on  an  observed  result.  The  mathematical  model  describing  the  observed  result  obtained  with 
different  values  of  Factors  A  and  B  is  given  by: 

Yigm)  =  H.  +  a.  +  Pj  +  (Ctp)ij  +  E.j(m)  (1) 


Y,j(m)  is  the  observed  result  corresponding  to  the  i*  and  j*  values  of  Factors  A  and  B,  respectively.  On  the  right-hand  side,  p  is 
the  mean  of  all  observed  results  in  the  data  set,  a,  and  P,  denote  the  individual  A  and  B  factor  value  effects,  (aP),j  represents  the 
effect  of  the  A-B  interaction,  and  e,j(m)  is  a  Gaussian  random  error  that  represents  an  unaccountable  contribution  to  Yy  The 
subscript  (m)  denotes  the  tn^  observation  of  YtJ. 

An  estimator  for  each  term  in  Eq.  (1)  is  found  in  terms  of  the  observed  results  using  the  method  of  least  squares  The  random 
error  e,j(m)  is  minimized  and  the  minimum  variance  estimators  are  computed  from  [11,14): 

«i  “  Yj.  -  |i.  Pj  =  Yj  -  |t.  ctP.j  =  Y.J  -  Y,  -  Y,  +fl  =  Y.j  -  Y.J  (2) 

where  a,  is  the  i1*1  least  squares  estimator  of  a, ,  Yj,  is  the  mean  value  of  the  observed  results  having  the  i1*1  value  of  Factor  A,  Yy  is 
the  mean  value  of  the  observed  result  (if  "m"  observations  are  available)  having  the  i*  and  j1*1  values  of  Factors  A  and  B  respectively. 
The  dot  notation  in  Eq.  (2)  is  defined  as  follows: 


iv  -  £  y.j 


Y,  -II' 
;»1 


In  Eq.  (3),  YtJ  is  summed  over  "a"  values  of  Factor  A  and  "b"  values  of  Factor  B.  For  example,  in  the  navigation  sensor  selection 
problem: 

FactorA,  NavaidType=[TACAN,  VOR.DME,  LORAN),  a=4  (Model  1) 

Factor  B,  Number  of  Ground  Statiions={One,  Two,  Three),  b=3 
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Substituting  the  least  squares  estimator  equations  into  the  model  in  Eq.  <  1),  the  deviation  of  an  observed  result  from  the  overall  mean 
is  given  by: 

[VyjO  =  +  (VjO  +  [Yj,- Yj  -Y,)  +  +  [V  Yj  (4) 

Deviation  Factor  A  Factor  B  A-B  Interaction  Residual 

from  overall  Contribution  Contribution  Contribution  Contnbuuon 

mean  to  total  to  total  to  total  to  total 

deviation  deviauon  deviation  deviation 

If  the  sums  of  squares  of  each  component  in  Eq.  (4)  are  computed,  the  total  variation  of  all  the  observed  results  ix  expressed  in  tem.s 
of  the  individual  factor,  interaction,  and  residual  vanauons: 

SStotal=  SSa  +  SSB  +  SSab  +  SSe  (5) 

where  SS  refers  to  the  sum  of  squares.  SStoTaL  is  the  total  variation  in  the  observed  result,  SSa  and  SSb  are  the  contributing 
variations  in  the  observed  results  "hen  values  of  Factors  A  and  B  are  compared,  and  SSab  is  the  contributing  variation  m  the 
observed  results  due  to  A-B  interactions  The  residual,  SSe,  is  the  unexplained  (random)  variability  in  the  result  when  observations 
with  different  factor  values  are  made.  In  the  NSM  exar  >ple  in  Model  1 ,  SStoTaL  is  the  total  variation  in  the  RSS  position  error  based 
on  12  simulations  averaged  oyer  the  entire  flight  time.  SSa  is  the  contributing  variation  in  the  RSS  position  error  due  to  using 
different  navaid  types,  SSb  is  the  contributing  vanauon  in  the  RSS  position  error  due  to  using  different  numbers  of  ground  stations, 
and  SSab  is  the  contributing  variation  in  the  RSS  position  error  due  to  interaction  effects.  Finally,  SSe  >s  the  variation  in  the  RSS 
position  error  that  cannot  b'  attributed  to  navaid  type,  the  number  of  stations,  or  interactions  between  these  two  factors. 

Equation  5  is  the  foundation  of  the  ANOVA  technique  When  each  sum-of-squares  component  is  computed,  many  conclusions 
concerning  the  factor  effects  can  be  made.  For  example,  if  SSa  >>  SSb  then  Factor  A  has  a  greater  effect  on  the  observed  result 
than  does  Factor  B.  The  F-test  determines  whether  Factor  A  s  apparent  effect  on  die  •'bserved  result  is  real  or  can  be  atmbuted  to  the 
residual  variation,  SSe-  Defining  the  F-ratio  for  Factor  A,  F0,  assuming  more  than  one  observed  result  (m  >  ’.): 


SSA/(a-l) 

SSE/(ab(m-l)) 


(6) 


The  sum  of  squares  for  each  source  of  deviation  is  normalized  by  the  degrees  of freedom  (DOF),  which  is  simply  one  less  than  the 
number  of  factor  values.  Since  Factor  A  in  Model  I  has  four  levels  (a  =  4),  then  the  DOFA  is  3,  and  fue  Mean  Squared  Error  of  A  is: 

MSA  =  5|a  (7) 


The  F-test  enables  a  choice  between  two  hypotheses  to  made  made  Staled  mathemaucally  for  the  NSM  problem,  the  hypotheses  are: 

H,:  The  RSS  errors  are  statistically  equal  for  different 
values  of  the  factor  under  consideration 
Ha:  The  RSS  errors  are  statistically  unequal  for  different 
value;  of  the  factor  under  consideration. 

H0  and  Ha  are  called  the  null  and  alternate  hypotheses  respectively.  The  null  hypothesis  suggests  that  there  are  statistically 
insignificant  differences  in  the  observed  results  for  the  factor  values  under  consideration.  The  alternate  hypothesis  suggests  the 
opposite  case  is  true.  When  the  null  hypothesis  holds,  the  F-ratio  follows  the  F-distnbution  [11);  when  the  alternate  hypothesis 
holds,  the  F-rauo  follows  a  complex  non-centra!  F-distnbution.  The  computed  F-ratio  is  compared  to  tabulated  values  that  indicate 
how  the  ratio  is  distributed,  allowing  a  hypothesis  to  be  selected.  Small  values  of  F  support  H0  whereas  large  values  support  Ha- 
Using  Factor  A  as  an  example,  the  decision  rules  needed  to  choose  the  hypothesis  for  a  specified  confidence  level,  (j,  are: 

If  Fa  5  Ftable  (£;  DOFa;  DOFe),  then  conclude  H^  (Model  2a) 

If  Fa  >  Ftable  (fc  DOFa;  DOFe),  then  conclude  HA  (Model  2b) 

£  identifies  the  "risk"  the  designer  is  willing  to  lake  that  the  F-test  will  reject  the  null  hypothesis  when  in  fact  the  null  hypothesis  is 
valid.  If  Ho  holds,  the  effect  of  Factor  A  on  the  result  is  statistically  insignificant,  whereas  when  Ha  holds,  the  effect  of  Factor  A  on 
the  result  is  statistically  significant.  Table  I  summarizes  the  computed  variables  needed  to  analyze  a  two-factor  ANOVA  experiment. 

When  two  observed  results  are  compared,  the  statistical  difference  between  the  results  is  determined  using  the  Scheffe 
method.  For  example,  in  a  navaid  ANOVA  experiment,  multiple  comparisons  of  radio  navaid  hybrids  lead  to  a  ranking  of  these 
systems  from  best  to  worst.  Two  results  are  statistically  unequal  if  their  computed  difference  exceeds  the  Scheffe  critical  difference, 
given  by: 

Dootcal  =[M$Efnf  +  Jr)  x  DOFfactor  r<  Ftable  (?;  DOFfactor:  DOFe)]1'2  (8) 

where  ni  and  nj  are  the  sample  numbers  of  the  two  results  being  compared.  Although  the  discussion  above  used  a  two-factor 
ANOVA  experiment  as  an  example,  the  theory  is  extendable  to  experimem  for  which  any  number  of  factors  are  to  be  investigated 
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Table  I  ANOVA  Table  for  a  Two-Factor  Experiment 


Source  of 
Variation 

Sum  of 
Squares 

Degrees  of 
Freedom 

Mean  Squared 
Value 

ftable 

Confidence 

Coefficient 

Factor  A 

SSa 

a*! 

SSA*a-I) 

MSA/MSE 

Ftable  (?;»-  l;»b(m-l )) 

(K)x  ioo% 

Factor  B 

ssb 

b-l 

SSbXM) 

MSB/MSE 

Ftable  (i);b-l;ab(m.l)) 

(1-Ox  100% 

A-B  Interaction 

SSab 

(a-lMb-l) 

SSAsA(a-IXb-D)  MSAB/MSE 

Ftable  ft(a-lXb-l);ab(m-l))  (K)x  100% 

Error 

ssb 

ab(in-l) 

SSe/Wm-l)) 

Total 

SStotal 

abm-l 

Identifying  Important  Factors  Using  ANOVA 

The  RSS  position  error  histones  from  over  200  covariance  simulations  were  obtained,  and  the  data  were  used  in  an  ANOVA 
four-factor  navatd  experiment.  The  goal  of  the  expenment  was  to  identify  which  of  the  factors  (navaid  type,  number  of  ground 
stations,  trajectory  effects,  performance  history)  and  their  interactions  had  statistically  significant  impacts  on  the  RSS  position  error. 
The  factor  values  used  in  the  ANOVA  experiment  were: 

Navaids=(  VOR,  DME,  LOR  AN,  TACAN,  GPS )  (Model  3 ) 

Number  of  Ground  Stations=(One,  Two,  Three) 

Trajectory  Type={High-Performance,  Commercial  Transport,  General  Aviation,  from  Fig.  1 ) 

Performance  History  =  (Intervals:  I,  II,  III,  IV) 

Since  each  trajectory  consists  of  four,  15-minute  legs,  the  Performance  History  (or  "time  interval")  factor  refers  to  the  RSS 
performance  obtained  within  each  15-minute  time  frame.  Four  single-station,  six  double-station,  and  four  triple-stauon  hybrids  were 
simulated  using  combinations  of  Stations  A-D  in  Fig.  1. 

The  ANOVA  summary  in  Table  II  shows  that  three  of  the  four  factors,  navaid  type,  number  of  aiding  ground  stations,  and 
performance  history,  are  "strongly  significant"  with  99%  confidence.  Scheffe  multiple  comparison  tests  were  applied  to  the  navaid 


Table  II  Analysis  of  Variance  Summary  and  F-test 
Results  for  Four-Four  Navaid  Experiment 


SOURCE 

OF 

VARIATION 

SUM  OF 
SQUARES 
(Sq.  N.MI.) 

DEGREES 

OF 

FREEDOM 

MEAN 
SQUARE 
VALUE 
(S<j  N.MI.) 

COMPUTED 

F-RATIO 

TABULATED 

F-RATIO 

CONFIDENCE 

COEFFICIENT 

MAIN  EFFECTS 

Between 

Navaids 

4  815E+02 

4 

I  2(W  E +02 

9  909E-+02 

3.320E+00 

99% 

Between 

Number 

2.423 E+02 

2 

1.21JE+02 

W3E+02 

4.610E-+00 

99% 

Between 

Trajectory 

6.234E-01 

2 

3117E-01 

2.566E+00 

2.300E+00 

90% 

Between 

Time  Intervals  5.729E+01 

3 

1.910E+01 

I.572E+02 

3.780E+00 

99% 

and  number-of-ground-station  factors  to  identify  the  specific  ditferences  within  each  groups;  for  example,  the  RSS  performance 
difference  between  GPS  and  TACAN,  all  other  factors  being  equal,  was  statistically  significant.  Referring  to  Tabic  III,  GPS 
provides  the  best  RSS  position  performance  when  compared  to  TACAN,  LORAN,  DME,  and  VOR;  this  is  consistent  with  the  results 
in  Fig,  2.  On  the  other  hand,  the  RSS  performance  difference  between  LORAN  and  TACAN  with  all  other  factors  being  equal,  was 
not  statistically  significant.  Ibis  means  that  a  LORAN  hybrid  could  perform  better  or  worse  than  a  TACAN  hybrid,  depending  on 
the  values  of  the  other  factors  (e.g.,  number  of  ground  stations).  As  expected,  the  comparison  test  concluded  that  VOR  and  DME 
give  higherposition  errors  than  either  TACAN  or  LORAN,  and  that  DME  performs  better  than  VOR.  The  multiple  comparison  test 
results  in  Table  III  yielded  the  same  performance  ranking  depicted  in  the  graphical  results  (e.g.,  Fig.  2),  while  utilizing  the 
information  content  of  a  large  number  of  independent  simulations. 

Further  investigation  into  the  ANOVA  interaction  effects  revealed  that  the  ranking  should  be  cautiously  applied  to  single-station  \ 

hybrids,  since  these  are  highly-sensitivc  to  trajectory  effects.  As  seen  from  the  magnitudes  of  the  computed  differences  in  Table  III,  ; 

double-station  hybrids  provide  much  smaller  RSS  position  errors  than  single-station  hybrids.  The  performance  difference  between  -> 


two  and  three  stations  is  not  as  dramatic  but  is  nonetheless  statistically  significant.  One  way  to  further  invesugate  the  sensitivity  of 
single-station  hybrids  to  trajectory  is  to  use  multiple  Scheffe  comparisons,  presented  in  Table  IV.  The  LORAN  and  DME  lesults 
clearly  show  that  aircraft  trajectory  significantly  affects  RSS  performance  when  single-station  INS  hybrids  are  used;  however,  the 
results  show  that  RSS  performance  using  double-  and  triple-station  INS  hybrids  is  less  sensitive  to  the  aircraft's  trajectory.  This 
result  verifies  die  RSS  performance  graphically  observed  in  Fig.  5.  Table  IV  also  provides  interesting  insight  into  the  behavior  of  the 
TACAN  and  VOR  systems.  TACAN  is  less  sensitive  to  trajectory  effects  than  LORAN,  DME,  and  VOR  for  single-stauon  hybrids 
because  TACAN  provides  distance  (p)  and  bearing  (8)  measurements  (i.e.,  p-0  navigation)  from  the  aircraft  to  the  station,  enabling 
a  position  fix  to  be  made.  In  contrast,  the  single-station  LORAN,  DME,  and  VOR  hybrids  each  use  only  one  measurement  to 
recalibrate  the  INS  so  that  a  position  fix  is  not  made.  However,  when  two  or  three  siauons  of  LORAN  or  DME  are  used,  a  position 
fix  is  provided  (p-p  navigation),  and  the  RSS  performance  becomes  less  sensitive  to  trajectory  effects.  This  explains  the  "damped" 
response  in  navigation  error  shown  in  Fig.  4. 

The  ANOVA  summary  in  Table  H  shows  that  the  factor  "trajectory”  is  weakly  significant  (90%  confidence).  Indeed  the  term 
"trajectory"  is  extremely  vague.  The  significant  differences  in  single-station  hybrid  performance  shown  in  Table  IV  and  Fig.  3 
suggest  that  trajectory  should  be  decomposed  into  attributes  that  describe,  in  better  detail,  what  these  effects  really  are.  The 
differences  between  the  high-performance,  commercial,  and  general  aviation  trajectories  (Fig.  1)  are:  distance  from  a  station, 
airspeed,  and  heading  with  respect  to  the  ground  stations.  These  factors  play  an  important  role  in  affecting  navigation  performance. 
In  summary,  the  ANOVA  and  Scheffe  methods  systematically  identified  trends  in  the  simulation  data  without  recourse  to  tedious 
graphical  analysis. 


Extracting  Rules  Using  Induction:  The  1D3  Algorithm 

The  ID3  Algorithm  uses  inductive  inference  to  extract  rules  [13]  from  an  example  set.  The  problem  space  is  described  in  terms  of 
attributes,  where  eac1  tribute  is  characterized  by  a  set  of  values  that  define  the  possible  "states."  For  example,  in  Model  3  ,  the 
navaid  type  and  nr  jc  of  ground  stations  were  shown  to  be  attributes  affecting  RSS  position  error.  The  attribute  values  for  the 
factor  "navaid  type '  were  (GPS,  LORAN,  TACAN,  DME,  VOR),  and  the  attribute  values  for  the  factor  "number  of  stations"  were 
[One,  Two,  Three).  The  ID3  algorithm  defines  the  shape  of  the  decision  tree  as  it  identifies  the  most  important  attribute  at  each 
decision  node.  The  algorithm  uses  an  Information-Theoretic  Measure  (ITM)  that  minimizes  the  number  of  tests  (attribute  nodes) 
necessary  to  define  the  decision  tree.  The  information  contained  in  an  attribute  is  quantified  using  the  attribute's  entropy  content, 
found  in  the  example  set  This  measure  is  the  information  gained  by  testing  the  attribute  at  a  given  decision  node;  it  is  defined  as  the 
difference  between  the  information  content  within  the  complete  example  set  and  the  information  content  within  the  attribute: 


rrMuttibutt  =  -  £  p(Cj)  In  p(c,)  +  £  fi:  £  P.J ln  P>j  (9) 

j-1  i-1  j*l 

where  p(Cj)  is  the  probability  of  occurrence  of  the  j*  result  in  the  total  example  set;  p,j  is  the  probability  of  the  i*  attribute 


Tabic  IV  SchcfK  Comparisons  of  Trajectory  Effects  on 

Single-,  Double-  and  Triple-Station  Hybrid  INSs 


KEY 

(♦)  Difference  u  Suu«ic*lJy  Sipuficini 
(•)  Diffrrence  it  SuosUcaP)'  hiigra/icant 


Sctefft  Critical  Difference  (95%)  *  0.22  N  Mi 

Naralda 

Mean  Differ 

N.  Ml.  - ^ 

LORAN 

TACAN 

DME 

VOR 

Siftflt-Statloa  Heb'Idi 

Between  Jet  Transport  and  OA 
Between  Jet  Transport  and  Hi|h-P 
Between  OA  and  Hjgb-P 

8  57  <♦> 

0  22  <♦) 
078  {♦) 

008  (•) 
007  <•> 

0  15  (•) 

091  (♦) 
022  <♦> 
0  70  (♦) 

to  <♦> 
01  ■  (■) 
l.l  <♦> 

Double-Station  Hjbridi 

Between  Jet  Ttanspor t  and  OA 
Between  Jet  Transport  and  ltifclvP 
Between  OA  and  liifh-P 

002S  (■) 

0  008  (-) 

0  34  (■) 

0  003  <•) 

oooo  (•> 

0  003  (■) 

OW  (•) 

0  02  <•) 
002  (■) 

0  56 

000  (.) 
056 

Trlple-Statioa  H/brlds 

Between  Jet  Tt  an  sport  and  OA 
Between  Jet  Transport  and  Htth-P 
Between  OA  and  Hi|h-P 

0009  (•> 
0000  (•) 
0009  <•) 

8  8  8 
o  o  o 

001  (■> 
ooo  (■> 
001  (•) 

0  36  <♦) 
002  <•) 

0  38  (♦) 

value  occuiing  with  the  result,  c,;  e,  is  the  total  number  of  examples  having  the  i1*1  attribute  value,  and  cp  is  the  total  number  of 
examples  in  the  training  set,  the  limits  M  and  N  denote  the  number  of  result  and  attribute  values  respectively. 

The  ID3  algorithm  uses  the  ITM  in  a  splitting  strategy  f  12J  to  decide  which  attribute  provides  the  most  information  from  the  example 
set.  When  several  attributes  are  tested  foi  their  information  con  ent,  the  one  giving  the  largest  ITM  value  (hence,  information)  is  the 
attnbute  chosen  The  ID3  algorithm  constructs  a  decision  tree  as  follows: 

1 .  The  root  node  is  chosen  using  the  ITM  values.  The  ITM  having  the  largest 
value  is  the  root  node,  and  the  examples  associated  with  the  root  node  are 
examined 

2.  If  the  examples  associated  with  the  current  node  all  have  the  same  class  value,  then  an 
endnodt  is  reached.  The  next  attribute  is  chosen  by  recalculaung  the  ITM  values  of  the 
remaining  attributes.  The  largest  value  determines  the  next  attribute. 

3 .  The  current  node  has  at  least  two  branches.  The  examples  arc  associated  with  their 
branches  according  to  their  values. 

4.  For  each  branch  node,  repeat  step  2,  until  all  endnodes  have  been  found. 

A  simple  example  us  ng  the  navigation  senior  management  problem  will  illustrate  how  decision  trees  are  formulated  using  the  ID3 
method.  Consider  tbit  ti  decision  tree  to  predict  RSS  position  errors  for  different  navaid  types  and  number  of  ground  stations  is  to  be 
extracted  from  a  set  of  covariance  simulations.  The  navm  1  types  and  numbers  studied  were: 

Navaid  Type  =  {LORAN,  TACAN,  VOR)  (Mode!  4a) 

Number  of  Ground  Stations  =  (Two,  Three)  (Model  4b) 

The  RSS  position  error  is  determined  from  five  simulations,  where  the  RSS  performance  for  each  simulation  has  been  classified  into 
four  categories: 


RSS  performance  classes  =  {c-1,  c-2,  c-14,  c-15) 


(Model  4c) 


where  the  values  c-1,  c-2,  c-!4,  and  c-15  represent  user-defined  intervals  of  RSS  position  error.  The  five  simulations  and 
corresponding  classified  RSS  errors  are  shown  in  Table  V: 


7'able  V  Training  Example  Set  for  Decision  Tree  Induction 


Navaid 

Number  of 

RSS  Position 

Type 

Stations 

Error  Class 

LORAN 

2 

c-2 

LORAN 

3 

c-1 

TACAN 

3 

c-2 

VOR 

2 

c-15 

VOR 

3 

c-14 

First,  the  ITMs  for  the  navaid  type  and  number  of  stations  attributes  are  computed.  In  this  example,  Nnavaid  =  3, 
Notations  =  2,  and  M  =  4.  Rom  Table  V,  the  probabilities  that  each  value  of  each  factor  occurs  with  each  of  the  RSS  position  error 
classes  is: 


Table  VI  Value-Result  Occurrence  Probab¬ 
ilities  for  Training  Example  Set 


e-1 

c-2 

c-14 

c-15 

LORAN 

PH=0.S 

P12=0.5 

pt3=0.0 

P!4=0-0 

TACAN 

P2i=0.0 

P22=l-0 

P23=0-0 

P24=00 

VOR 

P3 1=0.0 

P32=0.0 

P33=0.5 

P34=0.5 

c-1 

c-2 

c-14 

c-15 

Two 

pn=0.0 

P12=0.5 

Pt3=0-0 

pt4=0.5 

Three 

P2i=.33 

P22=-33 

P23=-33 

P24=0.0 

Using  Eq.  (9)  and  the  probability  tables  above,  the  ITSs  for  the  two  attributes  are: 


ITMnavaids  =  0,776  bus,  ITMsrxnoNS  =  0.394  bits 


From  step  one  of  the  ID3  algorithm,  the  navaid  type  attribute  is  chosen  as  the  first  node  because  its  1TM  value  is  larger  than  that  of 
the  number-of-stations  attribute.  Applying  the  remaining  steps  in  the  algorithm,  the  decision  tree  extracted  fron'  the  example  set 
above  would  look  like  the  following: 


Developing  the  ID3  Attribute  Framework  Using  ANOVA  Results 

The  Model  3  covariance  simulations  were  used  to  extract  decision  trees  foi  the  NSM  Expert  system.  Eleven  attributes  were 
defined  for  the  ID3  framework: 

•  Navaid  type, 

•  Trajectory  leg, 

•  Aircraft  groundspeed, 

•  Number  of  ground  stations, 

•  Minimum  geodetic  distance  from  station , 

•  Maximum  geodetic  distance  from  station, 

•  (Max  -  Min)  distance  on  trajectory  leg, 

•  Maximum  line-of-site  angle  from  station, 

•  Minimum  line-of-sight  angle  from  the  station, 

•  Direction  of  flight  (approaching  or  receding)  relative  to  station, 

•  RSS  position  error  class  on  previous  trajectory  leg. 

The  distance  from  a  ground  station  is  an  important  attribute  since  the  signal-to-noise  ratio  decreases  as  the  distance  to  the  station 
increases.  The  direction  of  flight  with  respect  to  the  station  influences  position  accuracy  through  its  effect  on  the  line-of-sight  angle. 

The  trajectory  parameters  were  computed  for  each  of  the  high-performance,  jet  transport,  and  general  aviation  trajectories  on  each 
trajectory  leg.  The  maximum  and  minimum  distances  to  the  aiding  station  were  also  determined  on  each  trajectory  leg,  in  addition  to 
the  difference  between  the  maximum  and  minimum  distances. 

When  more  than  one  station  was  used  the  maximum  and  minimum  distances  were  the  closest  and  farthest  distances  computed  to 
the  stations.  The  distance  difference  is  the  algebraic  difference  between  the  farthest  and  closes,  distances  determined  on  the  trajectory 
leg.  A  similar  definition  was  applied  to  the  line-of-sight  angle;  from  the  angles  computed  to  each  station,  the  largest  and  smallest 
were  selected.  The  1D3  algorithm's  task  was  then  to  determine  how  these  attributes  were  related  to  each  other  and  to  the  final  RSS 
position  error. 

The  classification  scheme  chosen  to  represent  the  RSS  position  error  endnode  in  the  NSM  decision  trees  is  depicted  in  Table  VII. 
Since  an  approximate  prediction  of  the  RSS  position  error  was  of  interest,  it  was  appropriate  to  represent  the  RSS  perfoimance  in 
terms  of  an  error  range. 
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Table  VII  RSS  Position  Error  Classification  Scheme 


(High-Accuracy]  [Medium-Accuracy]  [Low-Accuracy] 


Error  Range, 

Code 

Error  Range, 

Code 

Error  Range, 

Code 

N.  Mi. 

N.  Mi. 

N.  Mi. 

0.0-0.02 

c-1 

0.10-0.20 

c-6 

1.0-1.5 

c-15 

0.02-0.04 

c-2 

0.20-0.30 

c-7 

1.5-2.0 

c-16 

0.04-0.06 

c-3 

0.30-0.40 

c-8 

2.0-2.5 

c-17 

0.06-0.08 

c4 

0.40-0.50 

c-9 

2.5-3.0 

c-18 

0.08-0.10 

c-5 

0.50-0.60 

c-10 

3.0-3.5 

c-19 

0.60-0.70 

c-11 

3.5-4.0 

c-20 

0.70-0.80 

c-12 

4.0-4.5 

c-21 

0.80-0.90 

c-1 3 

4.5-5.0 

c-22 

0.90-1.00 

c-14 

>5.00 

c-23 

The  velocity,  distance,  and  line-of-sight  angles  were  expressed  in  terms  of  ranges  instead  of  individual  values,  so  that  the  expert 
system  weights  trends  more  heavily  than  specific  examples.  This  renders  the  expen  system  more  adaptable  to  new  conditions, 
because  matches  between  the  actual  and  knowledge-base  cases  could  be  obtained  more  frequently. 

The  example  set  was  developed  using  the  attribute  framework  described  above.  The  RSS  position  errors  for  each  simulation 
were  classified  on  each  trajectory  leg  using  the  scheme  in  Table  VII.  The  ID3  example  base  was  then  created  from  each  single-, 
double-,  and  triple-station  simulation. 

NSM  Decision  Trees 

The  NSM  example  set  was  divided  into  17  smaller  example  sets.  The  GPS  and  on-board  navaid  examples  were  grouped  into  one 
expert,  whereas  the  ground-based  navaid  examples  were  divided  according  to  navaid  type  and  time  (15-minute  intervals).  The  ID3 
algonthm  constructed  decision  trees  for  each  of  the  17  small  expert  systems  that  comprise  the  larger  NSM  Expert  The  breakdown  of 
the  NSM  Expert  into  smaller  systems  provides  greater  manageability  of  the  training  example  base.  The  total  number  of  examples 
used  to  develop  the  NSM  Expert  System  was  932,  based  on  260  Kalman  Filter  covariance  simulations.  An  additional  37  simulanons 
were  performed  to  obtain  a  decision  tree  to  estimate  RSS  performance  when  different  navaid  types  are  combined.  The  NSM  expert 
system  prompts  the  user  for  a  set  of  flight  conditions  commensurate  with  the  attribule/value  lists  used  in  the  example  set,  and  the 
resulting  RSS  classification  code  is  returned  to  the  user  from  the  decision  tree. 

A  typical  decision  tree  obtained  for  the  ground-based  navaids  is  exemplified  by  the  TACAN  results.  Figure  8  presents  the 
decision  trees  for  single-,  double-,  and  triple-station  combinations  on  the  first  15-minute  trajectory  leg.  Here,  the  majority  of  the 
testing  nodes  are  trajectory  parameters  (distance,  LOS  angle,  direction  of  flight  with  respect  to  the  stations).  The  top  or  root  node  in 
Fig.  8  is  the  aircraft's  direction  of  flight  This  is  expected  because  the  distance  and  LOS  angle  attributes  are  dependent  on  directional 
motion.  Distance,  LOS  angle,  and  groundspeed  are  results  of  the  aircraft's  motion;  hence,  they  represent  more  specific  problem 
parameters,  and  it  is  expected  that  these  parameters  appear  at  a  lower  depth  in  the  decision  tree.  Figure  8  also  shows  that  distance, 
ground  velocity,  LOS  angle,  and  hybrid  performance  history  are  significant  factors  that  enable  a  prediction  of  the  RSS  error  to  be 
made.  The  RSS  classification  results  verify  that  the  closer  the  aiicraft  is  to  a  station,  the  smaller  is  the  RSS  error;  other  results  show 
that  the  larger  is  the  LOS  angle,  the  smaller  is  the  RSS  eiror  [14], 


Figure  8.  Decision  Trees  Predicting  RSS  Position  Error  Range  for  an  INS 
Aided  by  TACAN  During  the  First  IS  Minutes  of  Flight 


The  expected  performance  of  the  GPS  system  on  each  trajectory  leg  is  shown  in  Fig.  9.  Note  that  the  aircraft's  groundspeed 
plays  an  important  role  in  the  GPS  hybrid's  performance.  Velocity  affects  the  measurement  dynamics  (history)  and  is  therefore 
classified  as  a  trajectory  effect.  From  Fig.  9,  the  two-satellite  hybrids  are  more  sensitive  to  these  velocity  effects  than  are  thwhrec- 
and  four-satellite  hybrids. 


Figure  9  Decision  Tree  Predicting  RSS  Performance 
for  an  INS  Aided  by  GPS 


Finally,  the  decision  tree  showing  what  position  error  range  is  expected  when  different  navaid  types  are  integrated  in  a  hybrid 
system  is  presented  in  Fig.  10.  Note  that  the  decision  tree  is  not  specified  for  a  given  trajectory  leg.  The  RSS  position  errors  for 
these  simulations  were  averaged  over  the  entire  flight  time  for  the  high-performance  trajectory.  The  tree  is  organized  in  terms  of  the 
navigation  method  used:  (1)  Distance-Velocity  (p-V),  (2)  Bearing- Velocity  (0-V),  (3)  Distance-Bearing  (p — 0),  (4)  Distance- 
Distance  (p-p),  (5)  Bearing  Bearing  (6-6),  and  (6)  Velocity-Velocity  (V-V).  These  results  show  that  LORAN  is  a  better  distance¬ 
measuring  navaid  than  DMfc  and  that  Doppler  Radar  provides  better  navigation  accuracy  than  the  Air  Data  Sensor  when  p-V 
navigation  is  used.  The  p-8  results  show  that  it  is  possible  to  obtain  good  performance  when  LORAN  and  VOR  are  used.  The 
LORAN/DME  hybrid  gives  better  results  than  two  DME  stations  but  worse  performance  than  two  LORAN  stations.  By  far  the 
worst  results  arc  obtained  using  two  VOR  stauons.  As  discussed  before,  the  VOR  system  is  the  least  accurate  measurement  device  of 
the  seven  systems  studied,  which  greatly  affects  INS-VOR  hybnd  results. 


Figure  10  Decision  Tree  Predicting  RSS  Performance  When 

Different  Navaid  Combinations  are  Used  to  Aid  an  INS 


PERFORMANCE  RESULTS  OF  NSM  EXPERT  SYSTEM 

It  is  important  to  quantify  the  NSM  Expert's  performance  for  several  test  scenarios.  It  also  is  desirable  to  determine  the  factors 
that  affect  the  system's  perfonnance,  so  that  these  factors  can  be  exploited  in  future  system  development. 

Two  high-performance  trajectories  were  used  in  the  performance  evaluation  of  the  NSM  Expert.  The  two  trajectories  each  consist 
of  four  15-minute  legs.  Trajectory  #2's  flight  pattern  was  in  a  counter-clockwise  direction,  whereas  clockwise  flight  patterns  were 
used  to  develop  the  NSM  Expert  (Fig.  1).  Additionally,  the  takeoff  point  on  Trajectory  #2  was  five  degrees  farther  north  than  the 
training  trajectories'  takeoff  points.  These  trajectory  differences  change  the  measurement  and  INS  dynamics,  affecting  the  hybrid 
performance.  Trajectory  #2  was  designed  this  way  intentionally,  so  that  the  NSM  Expert  System's  adaptability  could  be  determined. 
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Single-,  double-,  and  triple-station  combination  hybrids  were  simulated  on  each  test  trajectory  for  the  DME,  VOR,  TACAN,  and 
LORAN  systems.  The  combinations  were  formed  using  four  ground  stations  located  as  in  Fig.  1  with  respect  to  each  other. 
Additionally,  two-,  three-,  and  four-satellite  hybrids  were  simulated  on  the  test  trajectories,  as  were  Doppler  Radar  and  Air  Data 
sensor  hybrids.  In  total,  60  covariance  simulations  were  performed  for  the  two  test  trajectories. 


Test  Trajector,  Data  Preparation,  Performance  Metrics,  and  Results 

The  performance  results  for  each  of  the  60  simulations  were  classified  on  each  trajectory  leg  according  to  the  scheme  in  Table 
VII.  The  total  number  of  matches  was  counted  on  each  leg  of  each  test  trajectory  for  the  seven  navaid  types  studied.  A  match  was 
declared  Between  the  actual  and  predicted  RSS  classification  if  and  only  if  the  RSS  classification  codes  differed  by  one  or  less.  For 
example,  if  the  NSM  Expert  predicted  an  RSS  classification  code  of  6  whereas  the  covariance  results  determined  <•  performance  of 
Class  7,  a  match  was  declared.  A  match  also  would  have  been  declared  if  the  actual  performance  was  Class  5.  Since  the  NSM 
Expert  is  only  expected  to  estimate  a  hybrid's  performance,  it  is  allowed  some  room  for  error. 

The  NSM  Expert  System  was  run  488  nrres  in  order  to  determine  the  number  of  matches  for  each  system  on  the  test  trajectories. 
Figure  1 1  shows  the  NSM  Expert's  performance  in  predicting  the  RSS  position  error  for  each  hybrid  configuration.  The  predictive 
performance  metric  for  each  navaid  is  defined  as  the  percentage  of  number  of  matches  obtained  from  the  total  number  of 
combinations  tested  for  that  navaid.  The  matches  on  all  four  trajectory  legs  are  reflected  in  this  figure. 

The  NSM  Expert  performed  very  well  on  the  two  test  trajectories.  Figure  1 1  shows  that  the  NSM  Expert  correctly  predicts  the 
RSS  position  error  better  than  70%  of  the  time  on  test  Trajectory  #1.  The  system  required  only  the  trajectory  information  end  its 
knowledge  of  hybnd  system  performance  to  make  these  predictions.  However,  its  predictive  capability  on  test  Trajectory  th  is 
slightly  worse  for  the  LORAN  hybrids  (69%),  considerably  worse  for  the  VOR  (45%)  and  Air  Data  sensor  hybrids  (53%),  and 
identical  for  the  remaining  configurations.  Hence,  the  results  from  Trajectory  #2  suggest  that  additional  investigauon  into  trajectory 
effects  on  VOR's  and  Air  Data  Sensor's  performance  may  be  necessary. 

The  results  in  Fig.  1 1  are  encouraging  for  expert  system  designers.  We  have  shown  that  an  expert  system  can  be  designed  from 
data,  and  that  good  results  are  obtainable  even  from  relatively  small  training  sets.  The  total  number  of  examples  used  to  obtain  the 
NSM  decision  trees  was  slightly  less  than  one  thousand. 

Next,  we  investigated  how  the  NSM  Expert's  performance  could  be  improved.  Two  factors  affecting  the  Expert's  performance 
were  examined:  the  number  of  problem  attributes  selected  to  describe  the  problem,  and  the  number  of  examples  in  the  ID3  training 
set.  Figure  12  shows  the  NSM's  predictive  performance  metric  plotted  as  a  function  of  the  number  of  attributes  used  in  the 
knowledge  bases  of  four  navaid  expert  systems  The  complete  attribute  set  may  be  found  in  Ref.  14. 

To  study  the  effect  of  number  of  attributes  on  the  NSM  Expert's  performance,  the  number  of  attributes  used  in  each  of  the  17 
expert  modules  was  decreased  one  at  a  time  and  new  decision  trees  were  obtained.  The  "root  node,"  representing  the  most  important 
attribute  on  which  a  decision  tree  is  be  based,  was  eliminated  each  time  in  order  to  establish  a  systematic  method  of  attribute 
elimination.  The  results  in  Fig.  1 2  were  obtained  by  running  the  NS.M  Expert  each  time  an  attribute  was  deleted.  The  results  reflect 
the  cumulative  number  of  matches  for  both  test  trajectories  on  all  four  trajectory  legs;  in  total,  3,296  runs  of  the  NSM  Expert  were 
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Figure  11.  Performance  of  Navaid  Experts 
on  Test  Trajectories 


Figure  12.  Performance  of  Selected  Navaid  Experts  With  Increasing 
Number  of  Attributes  Describing  the  NSM  Problem  Space 


required  to  produce  the  results  in  Fig.  12.  As  seen  in  the  figure,  the  performance  of  each  navaid  expert  increases  as  the  number  of 
attributes  increases.  There  is  a  small  dip  in  the  DME  expert’s  predictive  RSS  error  capability  when  nine  attributes  are  used;  since  a 
relatively  small  number  of  training  examples  was  used  to  develop  the  NSM  expert,  small!  dips  can  be  expected,  and  they  are  relatively 
insignificant  when  compared  to  tire  overall  trend. 

Tne  effect  of  number  of  examples  in  the  ID3  training  set  on  the  NSM  Expert's  performance  is  shown  f  ;r  the  TACAN  Expert  in 
Fig.  13  for  both  test  trajectories.  This  figure  was  obtained  by  deleting  examples  in  the  training  set,  remducing  the  decision  trees  on 
each  time  interval,  and  counting  the  number  of  matches  between  covanance-determined  and  NSM-determined  RSS  classification. 

As  seen  in  the  figure,  there  is  an  increase  in  expert  system  performance  with  increasing  number  of  examples.  An  important  trend 
in  Fig.  13  is  the  reduced  rate  of  performance  increase  with  increasing  numbers  of  examples.  Although  it  is  difficult  to  extrapolate 
directly  from  the  curve,  it  is  estimated  that  an  80%  prediction  performance  could  be  attained  with  greater  than  300  TACAN  hybrid 
examples. 
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Figure  13.  Performance  of  TACAN  Experl  With 

Increasing  Number  of  Training  Examples 


Selecting  Navigation  Strategies  Using  the  NSM  Expert  System:  One  Navaid  Available 

The  NSM  Expen  was  used  to  determine  navigation  strategies  for  several  sensor  availability  scenarios  on  two  high-performance 
test  trajectories.  The  NSM-recommended  strategies  were  then  compared  with  strategies  obtained  from  examining  covariance  data.  In 
the  first  test,  the  NSM  Expen  was  limited  to  choosing  a  strategy  when  only  hybrids  of  one  navaid  type  may  be  used  or  when  only 
one  measurement  may  be  processed.  In  the  second  test,  the  NSM  Expen  chose  strategies  when  two  measurements  of  similar  or 
dissimilar  navaids  could  be  processed. 

In  Figs.  I4a-b,  the  NSM  Expen  found  the  best  navigation  strategies  on  both  test  trajectories  when  TACAN  Stations  A-D  are 
available.  For  test  trajectory  #1  shown  in  Fig.  14a,  the  NSM  Expert  recommended  Station  A  for  the  first  30  minutes,  Station  C 
from  30-45  minutes,  and  Station  B  from  45-60  minutes.  For  test  trajectory  #2  shown  in  Fig.  14b,  the  NSM  Expert  recommended 
Station  A  throughout  flight.  Similar  strategies  were  determined  when  DME  Stations  A-D  were  available.  This  is  because  Stations 
A-D  arc  fixed  with  respect  to  the  two  test  trajectories.  Therefore  the  relative  RSS  performance  for  the  stations  would  be  similar 
whatever  the  station  type  might  be. 
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Figure  14.  Comparison  of  NSM-Recommended  and  Covariance-Determined 
Nav  Strategies  when  One  Navaid  Type  Available  to  Aid  an  INS 


Comparisons  of  single-measurement  NSM-recommended  and  covariance-determined  strategies  are  shown  in  Fig.  15  when 
navaids  of  different  types  are  available.  Two  scenarios  were  investigated  in  these  figures,  as  follows: 

Scenario  A  -  DME  Stations  B  and  C,  L.ORAN  Station  D,  and  Air  Data  Sensor  are  available. 

Scenario  B  -  VOR  Station  C,  DME  Station  A,  and  LORAN  Station  A  are  available. 


-t'*'  A  • " 


25-15 


a)  Recommended  Strategy  with  One  Available 
Measurement  *  Scenario  A,  Trajectory  #  1 


b)  Recommended  Strategy  with  One  Available 
Measurement  •  Scenario  A,  Trajectory  #  2 


c)  Recommended  Strategy  with  One  Available 
Measurement  -  Scenario  B,  Trajectory  #  1 


d)  Recommended  Strategy  with  One  Available 
Measurement  ■  Scenario  B,  Trajectory  #  2 


Figure  IS.  Comparison  of  NSM-Recommended  and  Covariance-Determined 
Nav  Strategies  with  One  Measurement  from  Different  Navaids 
Available  to  Aid  an  INS 


On  test  trajectory  #1  when  Scenario  A  navaids  are  available,  the  NSM  Expert  recommended  that  LORAN  Station  D  be  used  until  the 
fourth  trajectory  leg  (Fig.  15a).  11160,  the  NSM  Expert  recommended  reconfiguring  the  hybrid  .'".iter  from  LORAN  Station  D  to  DME 
Station  C;  however,  the  computed  covariance  results  show  that  the  best  reconfiguration  would  be  from  LORAN  Station  D  to  DME 
Station  B.  The  resulting  difference  in  performance  is  slight.  From  Fig.  15b,  the  NSM  Expert  chose  LORAN  D  to  aid  the  INS 
throughout  the  entire  flight .  For  Scenario  B,  the  NSM  Expert  chose  LORAN  A  over  DME  A  and  VOR  C  to  aid  the  INS  on  both 
trajectories,  as  shown  in  Figs.  15c  and  15d.  This  suggests  that  the  NSM  Expen  "learned"  the  colocation  rule  exhibited  in  Fig.  2- 
When  two  stations  are  colocated,  the  selection  is  based  on  the  navaid  hierarchy  LORAN,  TACAN,  DME,  and  VOR 

Selecting  Navigation  Strategies  Using  the  NSM  Expert  System:  Two  Measurements  Available 

Using  the  navaids  available  in  Scenarios  A  and  B  above,  the  NSM  Expen  was  given  the  task  to  find  the  best  two-measurement 
hybrid  strategies.  Since  hybrids  were  to  be  formed  from  different  navaid  types,  the  navaid  mixing  decision  tree  in  Fig.  10  was  used 
in  addition  to  the  17  individual  navaid  experts.  Recall  that  Fig.  10  provides  an  approximation  of  the  expected  position  error  for  the 
various  navaid  mixes  and  does  r.ot  include  trajectory  and  performance  history  effects 

The  results  obtained  by  muting  the  Scenario  A  navaids  together  are  shown  in  Fig.  16.  On  both  test  trajectories,  the  NSM  Expert 
recommended  that  two  GPS  satellites  be  used  to  aid  the  INS.  The  second  and  third  choice  configurations  were  the  LORAN  A/DME 
A  and  Doppler  Radar  hybrids,  respectively.  From  Fig.  16b,  the  LORAN/VOR  hybrid  performed  on  a  par  with  the  Doppler  Radar 
hybrid  on  test  Trajectory  #2,  although  this  was  not  predicted  by  Fig.  10. 
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a)  Recommended  Strategy  with  Two  Available 
Measurements  -  Scenario  A,  Trajectory  #  1 


b)  Recommended  Strategy  with  Two  Available 
Measurements  -  Scenario  B,  Trajectory  #  2 


Figure  16.  Comparison  of  NSM-Recommended  and  Covariance-Determined 
Nav  Strategies  with  Two  Measurements  from  Different  Navaids 
Available  to  Aid  an  INS 


CONCLUSIONS 

The  performances  of  seven  navigation  systems  aiding  a  medium-accuracy  Inertial  Navigation  System  (INS)  were  investigated 
using  Kalman  Filter  covariance  analyses.  Hybrid  performance  decisions  were  based  on  the  RSS  position  error  history  memc.  A 
Navigation  Sensor  Management  Expert  System  was  designed  from  covariance  simulation  data  using  a  systematic  method  comprised 
of  the  two  statistical  techniques,  the  Analysis-of- Variance  (ANOVA)  method  and  the  ID3  algorithm. 

ANOVA  results  show  that  statistically  different  position  accuracies  are  obtained  when  different  navaids  are  used,  the  number  of 
radio  navigation  ground  stations  or  Global  Positioning  System  satellites  used  to  aid  the  INS  is  varied,  the  aircraft's  trajectory  is 
varied,  and  the  performance  history  is  varied.  By  indicating  that  these  four  factors  significantly  affect  the  decision  metric,  an 
appropriate  parameter  framework  was  designed,  and  a  simulation  example  base  was  created. 

The  example  base  was  composed  of  over  900  training  examples  from  nearly  300  simulations.  The  example  base  was  divided  into 
17  smaller  groups  to  enhance  manageability.  The  ID3  algorithm  then  was  used  to  determine  the  NSM  Expert's  classification  "rules" 
in  the  form  of  decision  trees  The  performances  of  these  decision  trees  were  assessed  on  two  arbitrary  trajectories,  by  counting  the 
number  of  times  the  rules  correctly  predicted  the  RSS  position  accuracy.  These  performance  results  then  were  presented  using  a 
predictive  metric. 

The  ANOVA/ED3  method  was  very  effective  for  the  systematic  development  of  the  NSM  Expert  using  simulation  data.  Results 
show  that  the  NSM  Expert  can  accurately  predict  the  expected  RSS  position  error  for  a  specified  navigauon  sensor  suite  operating  on 
a  specified  aircraft  trajectory  between  45%  and  100%  of  the  time.  The  test  trajectories  used  to  evaluate  the  system’s  performance 
show  that  the  NSM  Expert  adapts  to  new  situations  and  provides  reasonable  estimates  of  the  expected  hybrid  performance.  The 
system's  good  performance  with  relatively  few  examples  clearly  shows  how  the  ID3  algorithm  maximizes  the  information  content 
contained  in  the  example  base.  The  performance  results  strongly  suggest  that  operational  systems  can  be  designed  from  simulation  or 
experimental  data  using  the  ANOVA/ID3  method  for  knowledge  acquisiuon  The  systematic  nature  of  the  method  makes  it  a  useful 
tool  for  expert  system  designers. 

With  slightly  less  than  300  navigation  hybrid  simulations  used  to  develop  its  knowledge  base,  the  NSM  Expert  exhibits  a  good 
capability  for  finding  the  best  or  second-best  navigation  strategies.  The  most  imponant  aspect  of  the  system  is  its  development  using 
mathematical  models  and  digital  computer  simulations,  and  not  through  knowledge-extraction  using  human  expertise.  Also  important 
is  the  systematic  development  of  this  Expert  System  using  the  well-known  ANOVA  and  ID3  statistical  methods.  The  exercise  also 
showed  how  potentially  very  large  expert  systems  can  be  broken  down  into  several  small  experts,  thereby  facilitating  system 
maintenance  and  enhancing  futuredevelopment. 

It  was  shown  that  the  NSM  Expert's  ability  to  solve  navigation  sensor  management  problems  increases  with  the  number  of 
examples  used  in  its  training  set  and  with  the  number  of  problem  descriptors.  The  NSM  Expert's  good  performance,  given  its 
relatively  small  training  set,  is  very  encouraging.  It  demonstrates  that  large,  carefully-planned  simulation  experiments  can  be  used  in 
a  systematic  manner  to  develop  a  rully-opcrational  expert  system  with  a  designer-specified  performance  effectiveness. 
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Other  aerospace  applications  that  are  good  candidates  for  the  ANOVA/ID3  method  are  air  combat  pilot  strategies  from  simulation 
or  flight  test  data  and  air  traffic  control  solutions  to  multi-configuration  problems.  The  expert  system  design  methodology  also  is 
pertinent  to  problems  such  as  nuclear  reactor  control  strategies,  chemical  process  control  strategies,  automated  highway  driving,  and 
robotics  applications.  In  each  case  simulation  or  operational  experiments  may  be  executed  for  the  systematic  development  of  an 
expert  system  advisor. 
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ABSTRACT 

The  development  of  expert  systems  require  the  use  of  engineering  techniques  which 
can  be  used  to  efficiently  and  correctly  extract  the  domain  knowledge  resldt.it  within 
the  human  expert.  To  apply  these  techniques,  certain  conditions  must  be  met.  These 
conditions  are  that  the  candidate  expert  system  domain  must  be  suitable  for  Implementa¬ 
tion,  that  there  be  a  knowledge  engineer  with  a  certain  level  of  domain  knowledge,  and 
that  the  right  human  domain  experts  be  selected  In  the  expert  system  development  effort. 
This  paper  presents  a  semi- sequent i al  approach  to  development  of  techniques  which  can 
be  used  to  extract  the  knowledge  from  the  human  expert.  Presented  are  both  direct  and 
indirect  methods  which  a  knowledge  engineer  can  use  to  extiact  this  knowledge. 

1 NTRODUCT I  ON 

In  recent  years,  expert  systems  have  been  developed  for  a  wide  variety  of  subject 
domains  such  as  medicine,  engineering,  and  the  sciences.  With  these  developments, 
there  has  been  a  creation  of  techniques  which  serve  to  extract  the  domain  knowledge 
from  the  human  experts.  The  methods  used  to  extract  this  Information  from  the  human 
domain  experts  Is  commonly  referred  to  as  knowledge  engineering.  There  are  two  main 
approaches  used  In  knowledge  engineering.  First,  there  Is  the  object-attribute-value 
approach  which  uses  forward-chaining  from  the  Inference  engine  and  knowledge  structure 
facts  to  reach  a  designated  goal.  Second,  there  Is  the  Inferential  approach  which 
uses  If-than  rules  from  the  goal  and  backchains  through  the  knowledge  structure  to 
deduce  the  facts.  Knowledge  engineering,  still  in  Its  Infancy  as  a  field,  consists  of 
the  knowledge  acquisition  or  extraction  process  which  is  used  to  develop  an  expert 
system.  Consequently,  the  knowledge  extraction  techniques  In  existence  today  are 
largely  the  result  of  tne  emerging  technology  which  has  risen  out  of  the  development 
efforts  of  current  expert  systems. 

Before  an  expert  system  Is  developed,  the  knowledge  engineer  must  understand  that 

this  effort  Is  not  a  trivial  task.  He  must  also  determine  whether  the  candidate  domain 

Is  suitable  for  Implementation  into  an  expert  system.  In  making  this  determination, 

It  Is  Important  that  the  following  conditions  be  met: 

(1)  the  domain  must  be  stable  enough  and  precisely  defined  so  that  it  can 
be  decomposed  into  the  various  subdomains  which  are  resident  with  It. 

(2)  there  must  be  a  user's  requirement  for  having  such  a  system. 

(3)  there  must  be  human  domain  experts  from  which  the  domain  knowledge  can 

be  extracted. 

(4)  there  must  be  a  knowledge  engineer  with  an  Indepth  understanding  of  the 
subject  domain. 

(5)  there  must  be  some  engineering  tools  or  methods  with  which  the  knowledge 
engineer  can  extract  the  domain  knowledge  from  the  human  expert. 

The  most  difficult  of  these  tasks  Is  the  extraction  of  the  knowledge  from  the  human 
domain  expert.  The  knowledge  engineer  or  knowledge  extractor  must  design  methods  for 
revealing  the  complex  structures  and  processes  used  by  the  expert  In  solving  various 
types  of  problems.  This  engineer  must  understand  that  the  expert  has  knowledge  stored 
In  various  types  of  structures.  Some  of  these  types  of  structures  Include  Information 
stored  In  lists,  tables,  decision  logic  trees,  and  In  hierarchies  of  relationships 
whicn  consist  of  nested  categories  or  clusters. 

The  knowledge  engineer  must  also  recognize  that  an  expert  possesses  the  skill  to 

adapt  old  patterns  which  he/she  sees  applicable  in  a  new  problem  through  the  application 

of  known  good  rules  used  before.  In  this  case,  the  knowledge  engineer  should  obtain  a 
;  set  of  representative  problems  which  the  expert  Is  familiar  with  and  have  him/her 

■  articulate  the  steps  and  methods  used  In  problem  solving.  The  Information  which  the 

knowledge  engineer  extracts  then  consists  of  rule  bases  which  can  then  be  Implemented 
Into  an  entity  called  the  Inference  engine.  This  engine,  a  very  Important  component 
!  of  the  expert  system,  Is  then  modified  as  new  rules  are  defined  during  the  course  of 

;  user-interaction  with  the  prototype  expert  system. 

;  This  paper  presents  a  semi-sequential  approach  to  the  development  of  techniques 

i  which  can  be  used  to  extract  the  knowledge  from  the  human  expert.  Presented  Is  a 

[  description  of  the  procedures  and  techniques  In  developing  an  expert  system.  The 

discussions  Include  consideration  of  the  domain  suitability  for  an  expert  system,  the 
!  knowledge  engineer  requirements  for  effective  knowledge  acquisition,  the  selection  of 

1  the  domain  expert,  the  actual  direct  and  Indirect  methods  for  knowledge  acquisition, 

j  some  discussions  on  system  prototyping  and  user  Interfacing,  and  on  testing  and  knowl- 

i  edge  base  maintenance  of  the  developed  expert  system.  This  familiarization  can  be 


i 
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acquired  In  the  following  ways:  To  accomplish  this,  the  engineer  should: 

OOMAIN  SUITABILITY  FOR  AN  EXPERT  SYSTEM 

Before  beginning  the  development  effort  for  an  expert  system,  the  proponents  for 
system  development  must  determine  whether  the  proposed  domain  Is  suitable  for  Incorpo- 
atlon  into  an  expert  system.  This  Is  done  by  obtaining  surveys  from  well-known  experts 
In  the  domain  on  what  they  believe  are  the  areas  where  such  systems  are  needed  most. 

The  experts  should  also  be  able  to  identify  the  Intended  users  of  such  a  system.  For 
example.  If  a  development  proponent  was  looking  at  the  feasibility  for  developing  an 
expert  system  to  aid  physicians  In  diagnosing  a  certain  disease,  the  proponents  would 
seek  the  advice  from  specialists  In  the  domain  -area  and  have  them  Identify  areas  where 
they  believe  that  such  a  system  would  prove  of  some  Intrinsic  value  to  the  Intended 
user.  Following  this,  the  knowledge  engineer  determines  whether  the  domain  Is  stable 
enough  for  development  by  getting  additional  opinions  from  well-known  experts  on  the 
variations  or  conflicts  which  may  exist  In  how  processes  and  their  outputs  are  viewed 
within  the  domain.  If  the  domain  Is  continuously  changing  or  evolving,  then  develop¬ 
ment  may  have  to  be  deferred  until  the  domain  stabilizes  or  there  Is  a  majority  of 
agreement  on  how  the  domain  Is  viewed  by  the  experts.  After  establishing  the  stability 
of  the  domain,  the  knowledge  engineer  must  then  determine  whether  the  candidate  domain 
knowledge  can  be  extracted  given  the  current  technology  and  knowledge  engineering 
methods.  This  Is  Important  because  a  domain  may  be  good  for  system  development  but 
there  may  not  be  any  existing  methods  for  extracting  the  knowledge  from  the  expert  or 
the  knowledge  engineer  (or  candidate)  may  not  have  sufficient  domain  knowledge  to 
develop  a  plan  for  extracting  the  Information. 

KNOWLEDGE  ENGINEER  REQUIREMENTS 

The  knowledge  engineer  should  have  a  working- 1 evel  familiarity  of  the  domain 
which  is  candidate  for  Implementation  In  an  expert  system.  To  accomplish  this,  the 
engineer  should: 

(1)  attend  tutorial  sessions  with  some  doma'n  experts  to  ootain  a  general 
understanding  on  the  domain  concepts  and  technology, 

(2)  prepare  a  domain  knowledge  document  whlcn  contains  general  domain 
knowledge.  This  document  also  serves  the  purpose  of  being  used  as  a 
training  tool  for  project  members  which  later  join  the  team  and  Is 
valuable  because  It  contains  the  common  grammar  used  In  the  expert  sys¬ 
tem. 

(3)  read  as  many  domain  reference  materials  to  Include  any  literature  on 
past  expert  system  development  efforts  which  are  related  to  the  current 
effort.  For  example.  If  a  knowledge  engineer  were  trying  to  develop 
knowledge  for  development  of  an  expert  system  to  diagnose  a  certain 
disease,  he/sno  would  make  sure  that  he/she  has  read  all  pertinent 
literature  on  systems  ltke  MYCIN  (Reference  1),  EMYCIN  (Reference  2), 
and  INTERNIST- I  (Reference  3)  to  name  a  few. 

(4)  conduct  short  knowledge  acquisition  sessions  with  the  experts  on  a  time- 
available  basis. 

A  required  output  of  the  familiarization  phase  Is  that  the  knowledge  engineer 
should  prepare  a  paper  (which  contains  the  knowledge  document  Identified  in  (2)  above) 
which  Identifies  the  knowledge  base  In  domain  facts  and  rules  which  are  clear,  descrip¬ 
tive,  concise,  and  preferably  In  a  pseudo-English  format.  This  allows  for  ease  In  the 
Initial  prototyping  of  the  system,  the  ability  to  backtrack  if  necessary  with  complete 
Information/knowledge  available  to  determine  problem  areas,  and  make  It  easier  for  the 
expert  to  communicate  In  a  format  which  Is  known  to  him/her. 

SELECTION  OF  THE  DOMAIN  EXPERT 

Since  the  human  domain  expert  Is  the  most  Important  driver  for  development  of  the 
expert  system,  the  knowledge  engineer  needs  to  make  sure  that  the  experts  which  he/she 
select  for  the  knowledge  extraction  process  are  well-known,  acknowledged  experts  within 
their  domain  community.  The  engineer  also  needs  to  determine  whether  the  expertise 
lies  with  certain  Individuals  or  within  the  confines  the  expert  community.  He/she  also 
needs  to  establish  some  criteria  for  the  selection  process.  Other  Important  factors 
are  that  the  candidate  experts  must  have  the  following: 

(1)  have  current  experience  on  working  problems  within  the  domain  of  their 
expertl se. 

(2)  the  support  of  their  management  so  that  they  can  devote  considerable 
time  to  the  development  project. 

(3)  be  cooperative,  and  willing  to  allow  for  the  knowledge  engineer  to 
extract  the  resident  knowledge  through  a  variety  of  methods  (some  of 
which  might  seem  boring,  time-consuming,  and  perhaps  a  bit  contrived). 

(4)  see  the  potential  benefits  that  he/she  can  reap  by  having  a  companion 
advisor  to  aid  him/her  In  doing  some  of  the  routine,  detailed,  or 
can  then  allow  the  expert  to  work  on  areas  which  interest  him/her  be¬ 
cause  of  having  more  time  for  these  efforts. 

(5)  complete  confidence  that  such  a  system,  once  available,  Is  not  meant  as 
a  replacement  for  his/her  expertise. 
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The  knowledge  engineer  can  accomplish  this  by  selecting  a  panel  of  domain  experts  (not 
the  ones  from  which  the  knowledge  Is  to  be  extracted)  to  Identify  potential  candidates 
for  selection.  These  candidates  are  then  given  typical,  non-contrlved  problems  which 
are  common  within  the  domain  and  evaluated  based  on  their  observed  strengths  and 
weaknesses  as  experts.  The  evaluation  would  be  based  on  the  ultimate  recommendation  of 
the  panel  of  experts  but  with  final  judgement  made  by  the  knowledge  engineer.  Once 
these  potential  candidates  have  been  selected,  the  knowledge  engineer,  with  the  consul¬ 
tation  of  the  expert  panel,  must  evaluate  the  experts,  and  select  one  or  several  (In 
some  cases)  experts  for  the  project. 

KNCWLEOGE  ACQUISITION 

Before  actually  starting  the  knowledge  acquisition  phase,  the  knowledge  engineer 
must  make  the  assumption  that  this  phase  Is  the  single,  most  Important  phase  In  the 
development  of  Inference  strategies  because  it  Is  a  driving  variable  In  the  development 
of  the  main  part  the  expert  system,  the  Inference  engine.  The  engineer  must  also 
understand  that  developing  an  expert  system  prototype  can  take  several  months  to  a  year 
as  the  knowledge  base  Is  expanded  and  reformulated  many  times.  Therefore,  It  Is 
Important  that  the  knowledge  engineer  choose  an  approach  which  Is  flexible  enough 
to  allow  for  ease  In  prototyping  a  system  as  early  as  possible  while  at  the  same  time 
having  a  definitive  enough  format  which  is  easy  to  follow  In  backtracking  procedures, 
should  the  need  arise. 

Initially,  knowledge  engineer  should: 

(1)  organize  knowledge  acquisition  meetings  so  as  to  maximize  his/her  access 
to  the  expert  with  only  minimal  Interruptions. 

(2)  allow  the  meeting  attendees  access  to  the  Information  gained  and  the 
prototype  which  Is  developed  as  soon  as  possible. 

The  first  consideration  Is  based  on  the  recognition  that  the  expert's  time  Is  very 
valuable  and  that  this  time  Is  not  entirely  available  to  the  development  effort.  For 
this  reason.  It  Is  Important  to  hold  the  meetings  away  from  the  expert's  office  so  as 
to  avoid  Interruptions.  However,  the  knowledge  engineer  must  also  recognize  that  he/she 
might  want  to  see  the  expert  perform  his  tasks  In  his/her  own  habitat.  So  It  is 
Important  that  tradeoffs  be  established  to  avoid  possible  conflicts  and  maximize  the 
amount  of  time  which  the  knowledge  engineer  has  with  the  expert.  The  second  Is  to 
Insure  that  expert  system  can  be  run  as  the  prototype  Is  being  developed  from  the 
acquired  knowledge  to  checkout  the  parts  of  the  program.  In  this  way,  the  outputs  of 
the  prototype  are  evaluated  and  used  as  drivers  for  the  development  for  subsequent 
modifications  to  the  prototype.  The  knowledge  acquisition  process  follows  the  subpro¬ 
cesses  during  the  knowledge  acquisition  phase: 

(a)  the  experts  Is  given  or  Is  told  to  provide  typical  domain  problems  which 
he/she  solves  In  a  step-wise  approach. 

(b)  the  knowledge  engineer  acquires  the  knowledge  acquired  through  a  variety 
of  direct  and  Indirect  techniques  and  documents  the  observed  rules  and 
facts  in  the  knowledge  document  using  the  descriptive,  concise  pseudo- 
English  format.  These  techniques  are  discussed  in  the  following 
sections . 

(c)  from  the  1 nformati on/knowl edge  gained  In  the  Initial  problem-solving, 
the  knowledge  engineer  generates  new  problem  sets  (either  by  hand  or 
through  the  use  of  a  computer)  and  has  the  expert  solve  these  problems. 
These  problems  should,  over  the  course  of  several  Iterations,  converge 
on  the  fringes  or  edges  of  where  the  existing  rules  and  facts  are 
approaching  the  limit  applicability.  In  this  manner,  the  knowledge 
engineer  can  be  assured  that  he/she  has  covered  the  complete  spectrum 
of  problems  feasible  within  the  bounds  of  the  expert  domain. 

(d)  as  the  rules  are  acquired,  the  prototype  build-up  Is  started.  The  ex¬ 
pert  is  then  called  upon  tc  provide  critiques  and  analyses  of  the  cur¬ 
rent  state  of  the  prototype  by  reviewing  the  applicability  of  the  cur¬ 
rent  rules  given  the  generation  of  new  problems  which  he/she  must  con¬ 
sider.  Hopefully,  the  expert  continues  to  generate  new  problem  sets 
which  the  knowledge  engineer  can  observe  and  start  the  knowledge  acqui¬ 
sition  process  on. 

Throughout  the  entire  cycle,  the  knowledge  document  Is  continuously  reviewed  and 
updated  as  new  rules/facts  and  their  modifications  are  uncovered.  The  following  sub¬ 
sections  discuss  the  Individual  direct  and  indirect  methods  which  can  be  used  by  the 
knowledge  engineer  to  acquire  the  knowledge. 

KNOWLEDGE  EXTRACTION  METHODS 
DIRECT  TECHNIQUES 

The  direct  techniques  for  knowledge  acquisition  or  extraction  consist  of  Inteivlews, 
questionnaires,  simple  observation  procedures,  verbalized  or  thi n k 1 ng-out-loud  protocols. 
Interruption  analysis,  drawing  of  closed  curves,  and  Inferential  flow  analysis.  These 
methods  are  used  to  have  the  expert  make  known  the  domain  knowledge  through  the 
description  of  the  processes  used  In  solving  typical  domain  problems.  Each  one  of 
those  techniques  Is  further  described  below. 
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INTERVIEWS 

The  Initial  objective  of  the  knowledge  engineer  Is  to  get  the  expert's  full 
cooperation  at  all  times  so  that  the  resident  knowledge  can  be  harnessed.  This  requires 
that  the  expert  be  willing  to  share  his/her  knowledge.  Hence,  the  engineer  should 
select  only  those  experts  which  meet  the  criteria  previously  Identified.  In  addition, 
the  knowledge  engineer  must  make  the  expert  completely  at  ease  so  that  he/she  Is  willing 
to  be  Interviewed. 

The  Interviews  can  be  structured  or  unstructured.  However,  Interviews,  by  their 
very  nature,  are  very  time-consuming.  Therefore,  the  knowledge  engineer  needs  to 
maximize  the  knowledge  gathering  potential  of  these  Interviews  so  that  there  Is  no 
wasted  time  with  the  expert.  This  Is  done  by  having  a  very  well  organized  plan  for 
conducting  Interviews  with  the  expert.  The  Interviews  can  be  used  to  Identify  the  sets 
of  objects  and  relationships  used  In  the  performance  of  his/her  domain  tasks.  The 
engineer  needs  to  ask  a  mixed  style  of  questions  which  might  focus  on  a  particular 
aspect  of  a  problem  and  then  attempt  to  generalize  It  to  other  types  of  problems.  This 
can  be  done  by  noting  the  expert's  responses,  rules,  and  objects  which  can  be  examined 
for  generality  In  later  sessions.  The  engineer  also  needs  to  find  out  If  the  expert 
thinks  of  the  objects  In  special  relationships,  lists,  tables,  or  physical  spaces.  In 
addition,  the  engineer  must  find  out  any  rel at lonal/organi zat 1 onal  factors,  judgement 
processes,  problem-solving  methods,  and  solution  designs.  When  interviewing  multiple 
experts.  In  a  case  wnere  the  expertise  perhaps  lies  within  a  community  of  experts,  the 
knowledge  engineer  should  also  find  out  If  there  are  any  special  design  considerations 
which  the  community  uses. 

In  an  unstructured  Interview,  the  knowledge  engineer  can  let  the  expert  verbalize 
as  he  goes  through  a  problem.  This  is  hard  to  do  because  the  knowledge  engineer  might 
Interfere  with  the  expert's  thought  processes  by  his/her  mere  presence. 

The  knowledge  engineer  should  also  not  try  to  Impose  his/her  opinions  or  level  of 

domain  understanding  on  the  expert.  In  other  words,  the  engineer  should  not  try  to 

prompt  the  expert's  answers  by  Influencing  his/her  remarks.  This  could  result  In  getting 
the  expert  sidetracked  away  from  his/her  reasoning  path  and  might  mean  that  the  knowl¬ 
edge  which  could  have  been  obtained  Is  lost.  The  engineer  should  let  the  expert  talk 

even  If  the  current  subject  area  seems  momentarily  unrelated  to  the  main  purpose.  If 
the  expert  remains  for  an  extended  amount  of  time  on  these  areas,  the  knowledge  engi¬ 
neer  should  look  for  the  opportune  moment  (without  Interrupting)  and  gently  steer 
the  expert  back  on  discussion  of  the  problem  at  hand. 

The  knowledge  engineer  can  therefore  be  content  with  what  little  knowledge  the 
expert  might  verbalize  and  defer  the  verbalized  or  thl nkl ng-out-1 oud  protocol  (discussed 
later)  until  a  future  time.  In  any  event,  the  engineer  might  want  to  make  an  audiotape 
or  videotape  of  the  Interview  so  that  any  of  the  expert's  speech  pauses,  nongrammat 1  cal 
segments,  or  unintelligible  segments  may  be  reviewed  and  reconciled  with  the  copious 
notes  the  engineer  takes  during  the  interviews.  These  knowledge  sources  can  reveal  the 
Inference  making  processes  present  (Reference  4).  The  Interviews  should  be  conducted 
In  a  small,  quiet  room  with  only  the  minimum  essential  project  personnel  present.  In 
most  cases,  this  Includes  only  the  knowledge  engineer  and  the  expert.  However,  in  some 
cases,  the  engineer  might  want  to  have  several  experts  (not  more  than  say  three)  If  It 
Is  believed  the  knowledge  lies  within  the  community  of  experts.  (The  subject  of  the 
interview  of  several  experts  is  discussed  In  more  detail  below). 

The  structured  Interview,  on  the  other  hand,  Is  a  derivative  of  the  unstructured 
type.  In  this  case,  the  knowledge  engineer  uses  Information  which  was  gained  In  previous 
Interviews  or  sessions  and  dwells  on  the  expert's  familiar  tasks.  In  order  for  this  to 
work,  the  knowledge  engineer  must  make  an  Initial  pass  of  all  available  knowledge  (even 
If  general  In  nature)  which  has  been  analyzed  or  reviewed.  The  expert  then  goes  over 
the  data  one  entry  at  a  time  and  makes  comments  which  the  engineer  records.  This  method 
forces  the  experts  to  systematically  go  over  the  knowledge.  The  expert's  comments  can 
be  used  to  change  the  data  base  which  exists  as  new  Information  Is  made  available. 

This  Information  could  Include  (1)  the  creation  or  deletion  of  entries,  (2)  the  quali¬ 
fication  or  explanation  of  the  current  entries,  (3)  a  reorganization  of  the  hierar¬ 
chical  or  categorical  structures  of  the  data  base,  or  (4)  the  addition  or  deletion  of 
categories.  After  such  changes  are  made,  the  expert  Is  again  taken  over  a  review  of 
the  data  base  until  the  knowledge  engineer  believes  the  knowledge  has  been  extracted. 

In  some  cases,  it  might  be  necessary  for  the  knowledge  engineer  to  interview 
multiple  experts.  This  happens  when  the  knowledge  which  Is  desired  for  extraction  Is 
found  to  be  resident  within  the  community  of  the  experts  as  a  whole.,  For  this  situation, 
the  knowledge  engineer  might  want  to  have  a  single  residence  expert  act  as  a  consultant. 
The  consultant's  duties  are  to  make  sure  that  the  problems  which  are  given  to  the 
experts  are  not  contrived  and  present  all  the  Information  required  for  solutions  or 
elaboration.  The  consultant  does  not  participant  In  the  design  process  because  of  the 
danger  of  Inserting  bias  information.  However,  he/she  might  prove  Invaluable  In  that 
subproblems  are  then  given  to  each  expert  for  general  solution.  This  means  that  the 
experts  do  not  have  to  solve  the  problem  all  the  way  through  as  long  as  the  main  solution 
processes  are  identifiable  and  representative  of  what  Is  required  In  the  problem 
specifications  provided  to  them.  The  knowledge  engineer  then  examines  the  individual 
solution  processes  to  find  out  If  the  experts  used  the  same  strategy  In  terms  of 
decomposition  of  the  main  problem  Into  subproblems,  subproblem  solution  methodology, 
and  differences  In  specialization  among  the  experts. 
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QUESTIONNAIRES 

The  knowledge  engineer  can  also  use  questionnaires  to  derive  the  objects  of  a 
domain  and  their  relationships  and  uncertainties.  This  is  a  very  efficient  way  of 
acquiring  knowledge  and  Information.  It  offers  the  advantage  of  allowing  the  expert  to 
complete  the  questionnaire  in  a  relaxed  manner.  However,  the  questionnaire  needs  to  be 
properly  designed  to  insure  that  the  maximum  amount  of  knowledge/information  is  ex¬ 
tracted  from  the  expert.  The  questionnaire  should  be  designed  with  a  considerable 
amount  of  effort  expended  to  structuring  the  questions  such  that  they  elicit  a  response 
on  how  the  expert  solves  a  typical  domain  problem.  The  two  basic  types  of  question- 
aires  are  variable  and  relationship  elicitations.  These  types  are  discussed  below  and 
are  not  the  same  as  those  used  in  survey  research.  In  variable  elicitations,  the 
expert  is  asked  for  the  variables  which  he/she  uses  and  a  description  of  the  following:' 

(1)  the  variable  name. 

(2)  a  description  of  the  variable. 

(3)  the  type  of  values  that  the  variable  can  assume. 

(4)  the  size  of  the  values. 

(5)  the  range  of  the  values. 

(6)  whether  there  is  any  uncertainty  in  the  values  assumed. 

(7)  whether  the  variable  is  known  at  the  start  or  uncovered  as  the  reasoning 
progresses. 

In  relationship  elicitations,  the  expert  is  asked  to  state  his/her  beliefs  on 
what  the  relationships  between  factors  are.  For  example,  the  expert  might  be  asked 
about  the  stock  market  wheth°r  there  are  any  relationships  between  the  price  of  gold, 
the  rumber  of  shares  traded  in  a  single  day,  and  the  prime  lending  Interest  rate. 

Through  this  process,  the  expert  is  asked  to  specify  the  possible  relationships. 

Once  the  methods  described  above  are  conducted,  the  knowledge  engineer  should  ask 
the  expert  to  scale  his/her  responses  for  any  uncertainty  in  the  particular  Inferences 
which  might  have  been  reported.  The  method  ov  using  questionnaires  has  justification 
and  is  appropriate  because  the  normal  verbal  response  obtained  from  the  expert  might 
not  contain  reliable  information.  The  reason  for  this  is  tnat  the  expert  may  not  be 
good  at  estimating  the  problem.  Often,  he/she  might  perceive  the  problem  to  be  very 
difficult  before  he/she  starts  the  problem-solving  process,  and  will  have  thought  that 
the  problem  is  trivial  when  finding  out  how  easy  it  was  to  solve.  In  some  cases,  the 
expert  might  show  extreme  conservatism  in  his/her  responses  and  therefore  cloud  the 
true,  underlying  answers.  In  other  cases,  the  expert  might  be  overly  optimistic  that 
what  he/she  are  conveying  to  the  knowledge  engineer  is  correct  and  factual. 

The  rating  scales  which  are  used  to  exact  the  information  from  the  expert  consist 
of  the  bar  scaling  and  the  five-point  verbal  (or  Meister)  scale.  With  the  first,  the 
expert  is  asked  to  approximate  his/her  level  of  uncertainty  from  a  range  of  levels 
reflecting  total  uncertainty  to  total  certainty.  With  the  second,  the  expert  is  asked 
to  apply  a  qualitative  rating  on  his/her  uncertainty  through  the  use  of  the  terms 
COMPLETELY,  REASONABLY,  BORDERLINE,  MODERATELY,  and  EXTREMELY  to  the  uncertainty. 

OBSERVATION  OF  TASK  PERFORMANCE 

The  knowledge  engineer  car  also  determine  how  the  expert,  makes  his/ner  judge¬ 
ments,  diagnoses,  or  design  decisions  when  working  through  a  problem,  in  order  to  do 
this,  the  engineer  must  look  at  the  expert  solve  sotie  typical  domain  problems.  Through 
observation  the  engineer  can  discover  the  objects,  relationships,  and  inferences  which 
the  expert  uses.  The  only  problem  Is  that  the  engineer  must  make  sure  that  the  Infor¬ 
mation/knowledge  is  correctly  recorded  so  that  the  domain  knowledge  is  captured  in  its 
entirety.  The  two  methods  wfticn  can  be  used  to  do  this  are  for  the  knowledge  engineer 
to  take  copious  notes  and  obtain  audiotape  recordings  (video  recordings  also  if  time 
and  the  development  budget  permit)  as  the  expert  articulates  tne  processes  he/she  used 
in  problem  solving.  This  method,  however,  is  burdensome  on  the  engineer  in  that  the 
knowledge  engineer  might  be  pressured  to  try  to  capture  every  possible  step  which  the 
expert  demonstrates  or  articulates.  The  end-results  could  be  that  the  enqtneer  might 
miss  significant  portions  of  the  processes  which  are  described  or  might  read  too  much 
into  what  the  expert  is  really  conveying.  The  videotaping  (and  to  a  certain  extent 
the  audlotaping)  processes  offer  the  flexibility  of  allowing  the  knowledge  engineer  to 
review  the  tapes  later  and  obtain  answers  to  what  he/she  missed  taking  notes  on. 

However,  the  disadvantage  Is  that  the  knowledge  engineer  might  rely  too  heavily  on  the 
tapes  and  will  not  take  down  notes  on  Important  facts  which  the  expert  conveys  while 
going  through  the  problem  solving  procedures.  Also,  this  method  should  be  applied 
expeditiously  because  in  some  cases  it  relies  too  much  on  assuming  that  the  expert  may 
be  stating  his/her  true  line  of  reasoning  in  the  process.  This  might  not  be  the  case. 

PROTOCOL  ANALYSIS 

Another  technique  which  is  closely  related  to  observation  of  task  performance  is 
protocol  analysis.  This  technique  Is  not  applicable  to  all  kinds  of  tasks  (Reference 
5).  The  difference  between  observation  techniques  and  this  technique  Is  that  the  expert 
performs  the  domain  task  by  going  through  an  applicable  problem  and  thinking  the 
processes  used  out  loud  as  he/she  does  so.  Normally  these  types  of  sessions  are  videotaped 
and  the  knowledge  engineer  annotates  the  displayed  behavior  after  the  session  by 
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reconciling  the  behavior  with  the  notes  taken  In  real-time.  Typical  entries  noted  on 
these  annotations  are  statements  about  what  the  expert's  goals  are  at  particular  steps 
In  the  problem-solving  process,  the  problem-solving  methods  used  by  the  expert,  and 
statements  about  the  expert's  cognition  of  the  processes  he/she  Is  using. 

This  method  offers  the  advantage  of  letting  the  knowledge  engineer  see  the  Infer-' 
ences  made  by  the  expert  about  objects,  their  relationships,  and  other  information 
from  the  session  transcripts  or  notes.  It  also  has  the  advantage  over  task  performance 
observation  In  that  there  Is  relatively  no  delay  between  the  act  of  thinking  and  the 
recording  of  the  act.  In  this  manner,  the  knowledge  engineer  details  tne  kinds  of 
tasks  the  expert  performs  by  noting  (1)  those  tasks  where  verbalization  Is  a  natural 
part  of  the  thought  process  or  that  verbal  Information  Is  produced  while  someone  makes 
Inference  on  them,  or  (2)  In  Identifying  the  features  of  the  objects  In  the  situation. 
However,  this  method  Is  not  applicable  to  all  types  of  tasks.  Tasks  In  which  Idiosyn¬ 
cratic  language  processes  are  used,  are  not  well-defined  enough  to  enable  knowledge 
extraction.  An  example  of  an  area  where  such  would  be  the  case  would  be  in  the  arts 
(such  as  music).  In  addition,  some  tasks  cannot  be  described  by  natural  language 
descriptions.  Domains  which  Involve  perception  or  motor  tasks,  .lo  not  lend  themselves 
to  these  types  for  tasks.  The  knowledge  engineer  must  also  consider  that  m  some  type 
of  tasks,  the  technique  of  verbalizing  might  distract  the  expert  Into  subtasks  which 
are  normally  used  only  as  a  side-consideration  by  the  expert.  In  this  case,  the  ex¬ 
pert  might  bt  channeled  down  a  path  which  deviates  from  the  correct  problem-solving 
reasoning  path.  Once  obtained,  the  protocol  analyses  must  be  examined  to  determine 
If  there  are  any  changes  In  the  attention  focus  used  by  the  expert.  Also,  they  are 
examined  for  identification  of  the  kinds  of  objects  which  the  expert  sees  In  the 
process  and  the  relationships  and  Inferences  that  he/she  sees  between  protocols. 

INTERRUPTION  ANALYSIS 

When  the  knowledge  engineer  no  longer  understands  the  expert's  thought  processes, 
the  engineer  can  use  a  technique  called  Interruption  analysis.  The  knowledge  engineer 
then  asks  the  expert  to  furnish  details  on  what  he/she  did  and  why.  By  doing  this,  the 
engineer  Is  trying  to  capture  the  specific  momentary  focus  and  Inferences  which  the 
expert  Is  using.  This  method  Is  very  instructive  about  the  processes  being  observed  at 
the  moment.  However,  the  drawback  is  that  once  the  expert  Is  Interrupted,  there  Is 
very  little  chance  of  getting  the  expert  to  continue  at  precisely  the  place  he/she  was 
prior  to  the  Interruption.  Consequently,  there  Is  a  propensity  for  the  knowledge 
engineer  to  lose  Information/knowledge  with  this  technique.  Therefore,  this  technique 
should  be  used  sparingly  and  only  as  a  last  resort.  It  is  most  applicable  and  offers 
the  most  promise  If  It  Is  applied  to  the  expert  system  prototype  when  comparing 
performance  with  the  human  expert. 

0RAW1NG  OF  CLOSED  CURVES 

A  specialized  technique  which  can  be  used  to  Indicate  the  relationship  among 
objects  which  are  In  some  physical  space  configuration  is  called  the  drawing  of  closed 
curves.  With  this  technique,  the  expert  is  asked  to  Indicate  which  collection  of  a  set 
of  objects  go  together,  and  to  draw  related  objects  In  a  closed  curve.  This  technique 
Is  applicable  to  any  spatial  representation  such  as  the  mapping  of  the  possible  position 
locations  on  a  game  board.  In  effect,  the  knowledge  engineer  Is  looking  for  the  aspect 
of  the  responses,  the  position  currently  displayed,  and  the  order  which  matches  a  closed 
curve , 


INFERENTIAL  FLOW  ANALYSIS 

A  technique  which  Is  a  variant  of  the  Interview,  Is  known  as  Inferential  flow 
analysis.  This  technique  Is  ad  hoc  In  nature,  simple  to  use,  and  displays  to  the  expert 
the  aspects  of  expertise  which  have  been  uncovered  at  a  certain  time.  It  can  also  be 
used  to  stimulate  the  expert's  mind  on  the  subjects  to  be  covered  on  future  interviews. 
Typically,  the  expert  Is  asked  questions  about  the  causal  networks  amplifying  the 
objects  of  concepts  In  the  domain  of  expertise.  The  knowledge  engineer  looks  for  a 
list  of  key  objects  of  expertise  and  then  asks  the  expert  to  cite  any  relationships 
which  he/she  sees  between  two  seemingly  related  objects.  The  answers  could  reveal 
linkages  among  Items  which  are  linked  In  an  Inverse  relationship  and  which  may  have 
another  Intervening  variable.  The  answers  should  also  uncover  some  consistency  In  the 
relationship  between  the  Intervening  variables.  Each  time  a  specific  Item  Is  mentioned 
In  the  expert's  answer.  It  Is  linked  with  the  other  Items  In  the  answer  with  a  relative 
link  labeled  either  positive  or  negative.  The  linked  Items  are  then  joined  In  an  all- 
inclusive  network  of  relations.  Normally,  the  link  Is  started  with  a  standard  weight 
and  strengthened  with  each  successive  mention  during  the  development  of  the  network. 

INDIRECT  METHODS 

The  Indirect  methods  for  knowledge  acquisition  do  not  rely  on  the  expert's  ability 
to  articulate  the  knowledge/lnformatlon  which  he/she  uses  In  the  solution  of  problems 
In  the  domain.  These  methods  exploit  other  types  of  behaviors  within  the  domain.  The 
bases  for  these  methods  lie  on  the  recalled  or  scaled  responses  from  which  the  knowledge 
engineer  can  make  Inferences  about  what  the  expert  must  have  known  In  order  to  respond 
In  the  way  he/she  did.  In  some  Instances,  the  expert  Is  completely  unaware  of  the 
mental  or  knowledge  processes  which  he/she  uses  to  obtain  some  very  complex  answers. 

With  these  methods,  the  expert  Is  not  asked  to  express  the  knowledge  directly  but  Is 
given  a  variety  of  tasks.  From  the  results,  the  knowledge  engineer  Infers  the  underlying 
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structure  among  the  objects  rated.  This  technique  makes  different  assumptions  about 
the  form  of  the  understructure  which  may  consist  of  physical  spaces,  lists,  networks,  or 
tables.  The  knowledge  engineer  must  use  only  those  methods  from  which  the  assumptions 
fit  the  expert's  best  guesses.  This  Information  is  obtained  through  the  Initial  Infer¬ 
ences  through  careful  questioning  and  noting  of  objects,  names,  and/or  notations  which 
the  expert  makes.  The  methods  discussed  below  consist  of  (1)  multt-dlmenslonal  scaling, 
(2)  hierarchical  clustering,  (3)  general  weighted  networks,  (4)  ordered  trees  from 
recall,  and  (5)  repertory  grid  analysis.  All  th  se  techniques  have  been  validated  in 
experimental  studies  and  have  shown  psychologlce1  validity. 

MULT  1-0 1 HENS 1 ONAL  SCALING 

A  method  which  allows  for  similarity  judgements  to  be  made  on  all  pairs  of  objects 
and  concepts  In  the  domain  Is  referred  to  as  multl-demenslonal  scaling.  This  technique 
Is  easy  to  use  but  the  Interpretation  of  Its  results  are  not.  The  technique  produces  a 
layout  of  the  Items  In  space.  From  this,  a  similarity  judgement  matrix  Is  derived 
where  the  objects  are  paired  off  and  compared  to  each  other  to  determine  their  relative 
similarity  with  each  other.  The  matrix  Is  then  used  as  an  Input  to  an  analysis  program 
which  searches  for  the  best  placement  of  the  object  In  the  user-defined  dimensional 
space.  Each  dimension  solution  has  an  association  factor  measurement  which  Is  a  total 
distance  differential  from  the  perfect  fit  association.  The  knowledge  engineer  then 
looks  for  solutions  which  have  distance  factors  which  are  small  and  aggregates  dimensions 
for  the  smaller  factors.  In  doing  this,  less  dimensions  are  plotted.  The  plots  are 
then  examined  to  Judge  the  best  placement  of  the  axes  and  the  appropriate  labeling  for 
them.  These  plots  are  then  mapped  as  sol ut 1 on-to- sol ut 1 on-to-slml 1 ar 1 ty  matrix  types. 

The  advantage  of  this  technique  Is  that  It  can  derive  Interesting  clusters  of  objects, 
the  distance  factors  for  the  closer  lying  objects,  and  the  outliers.  The  difficulty 
with  this  technique  Is  that  It  Is  a  tedious  focess  which  requires  multiple  collections 
of  pair-wise  similarity  judgements,  it  Is  a' so  difficult  to  see  the  dimensionality 
with  the  smallest  dtstance  factor  and  then  perceive  the  best  placement  of  the  axes  and 
their  names.  The  judgements  are,  for  the  m.'St  part,  assumed  to  be  symmetrical  and  to 
have  a  value  from  zero  to  one.  This  roethob  Is  also  good  for  producing  a  diagram  which 
the  expert  can  Inspect  and  describe  In  mo' o  detail  as  the  knowledge  engineer  asks  more 
questions. 

HIERARCHICAL  CLUSTERING 

A  simplified,  straight  forward  algorithm  which  consists  of  half-matrix  similarity 
judgements  Is  known  as  hierarchical  clustering.  The  underlying  assumptions  for  this 
algorithm  are  In  direct  contradiction  to  those  for  multi-dimensional  scaling  (multi¬ 
dimensional  scaling  assumes  a  symmetrical  distribution  of  graded  properties  whereas 
hierarchical  clustering  assumes  that  the  Item  Is  not  a  member  of  the  cluster).  The 
judgements  are  made  as  a  function  of  'he  nested  clusters  which  two  Items  have  In  common 
or  the  height  at  which  two  Items  beco...e  members  of  the  same  category.  To  start  the 
algorithm,  the  knowledge  engineer  1 n' t ‘ al 1 zes - the  half-matrix  set  by  entering  the 
distances  between  Items.  This  results  In  a  hierarchical  representation  of  the  Items 
where  the  pairs  of  Items  closest  1r.  th-  matrix  are  joined  and  assumed  to  be  members  of 
the  same  cluster.  Then  a  new  matrix  Is  drawn  up  with  one  cluster  serving  as  a  new 
Item.  The  new  matrix  Is  then  reexamined  to  find  Items  which  are  closer  together.  Each 
time  a  new  matrix  Is  drawn,  the  Int.  r-ltem  distances  between  unclustered  Items  are 
calculated  to  determine  the  distance  factors  (minimum  distance  of  all  cluster  Items  to 
each  Item,  and  a’so  the  maximum  and  average  distance  factors).  These  values  are  then 
copied  from  the  original  matrix  onto  the  new  matrix.  The  advantage  of  using  this 
technique  Is  that  It  can  be  done  through  hand-calculations  which  are  easy  to  follow. 
However,  the  procedure  Is  tedious  because  you  have  to  collect  the  Items  as  In  multi¬ 
dimensional  scaling  and  without  some  valid  justification  for  employing  a  particular 
joining  algorithm,  the  enjoining  process  Is  somewhat  arbitrary.  The  result  Is  that  you 
could  derive  different  algorithms  for  the  different  hierarchies  and  the  analysis  Is 
very  subjective. 


GENERAL  WEIGHTED  NETWORKS 

Like  with  multi-dimensional  scaling  and  clustering  techniques,  the  general  weighted 
network  technique  gives  the  knowledge  engineer  a  way  of  extracting  the  expert's 
symmetrical  distance  judgements  on  all  possible  object  pair  combinations.  The  distance 
Is  assumed  to  arise  from  the  expert  traversing  a  network  of  associations  where  the 
network  Is  one  In  which  the  single  primary  path  between  every  two  Items  and  from  some 
difference  enclosed  for  the  secondary  path.  From  the  distance  matrix,  the  knowledge 
engineer  derives  the  minimally  connected  network  where  each  connection  must  involve 
closely  connected  or  linked  Items.  Then  additional  links  are  added  to  the  tree  and  the 
structure  is  referred  to  as  the  minimally  elaborated  network.  From  that  point,  a  link 
Is  added  If  and  only  If  It  Is  shorter  than  links  currently  serving  the  network  between 
those  two  nodes.  The  minimally  connected  and  minimally  elaborated  networks  are  then 
examined  for  any  dominating  concepts  with  large  numbers  of  connections  to  many  nodes 
and  for  Items  which  are  linked  Into  circles.  This  technique  can  then  be  used  to  reveal 
significant  aspects  of  expertise  which  should  be  encuded  into  the  expert  system. 

ORDERED  TREES  FROM  RECALL 

A  method  which  Is  used  to  Investigate  the  memory  organization  of  the  expert  Is 
commonly  referred  to  as  ordered  trees  from  recall.  This  method  has  been  Investigated 
to  show  the  relative  differences  on  memory  organization  between  experts  and  novices  In 
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a  domain.  The  ordered  trees  begin  with  recall  trials.  This  method  assumes  that  objects 
either  belong  to  a  cluster  or  do  not  like  In  the  hlarchlcal  clustering.  However,  a 
difference  1$  that  It  Is  built  on  a  model  of  how  the  data  Is  produced  by  a  subject. 

This  method  also  assumes  that  people  recall  all  Items  which  are  from  a  stored  cluster 
before  recalling  Items  from  another  cluster.  The  result  Is  that  the  technique  builds 
on  data  from  recalled  Items  from  known  or  learned  organizations.  The  similar  patterns 
which  are  found  to  exist  over  a  set  of  recall  orders  are  then  assumed  to  be  Indications 
of  organized  memory.  The  analysis  of  these  patterns  can  be  done  manually  but  Is  ex¬ 
tremely  tedious  and  subject  to  perceptual  error.  Hence,  computer  analysis  Is  best. 

This  technique  Is  used  most  In  studies  where  expert  and  novice  domain  practitioner 
differences  are  being  investigated.  Experts  usually  show  much  more  similarity  In 
their  organization  than  do  novices.  They  also  show  more  of  a  trend  toward  convergence 
In  their  methods.  Once  the  ordered  sequences  or  trees  are  obtained,  the  knowledge 
engineer  must  then  examine  them  and  determine  what  and  how  the  expert  perceives  a 
situation  resident  within  his/her  domain.  Reltman  and  Ruetar  (Reference  6)  show  good 
examples  of  how  to  apply  this  technique. 

REPERTC  ,Y  GRID  ANALYSIS 

A  technique  which  Is  based  on  clinical  psychology  construct  theory  Is  known  as 
repertory  grid  analysis.  This  technique  has  been  used  to  develop  several  hundred 
prototype  systems  and  Is  thought  by  some  expert  system  developers  to  be  the  most 
complete  technique.  The  technique  Is  applicable  to  classification  type  problems  where 
new  problem  observations  and  the  objects  are  sorted  into  one  known  set  of  categories. 

An  Initial  dialogue  Is  established  with  the  expert  through  an  unstructured  Interview. 

The  expert  Is  then  asked  to  name  objects  In  the  domain.  Once  this  Is  done,  the  expert 
Is  asked  to  distinguish  the  traits  between  objects  and  to  name  the  dimensions  of  the 
objects  In  his/her  own  words.  The  expert  is  then  told  to  Indicate  the  high  and  low 
values  for  the  objects.  The  knowledge  engineer  then  records  the  dimensions  and  scales 
provided  by  the  expert  for  three  Items  taken  at  a  time  (triples)  until  the  major 
dimensions  of  the  similar  and  dissimilar  objects  are  uncovered.  This  forms  a  grid  of 
objects  and  the  associated  dimensions.  The  knowledge  engineer  then  asks  the  expert  to 
fill  In  all  the  missing  values.  The  Initial  dialog  Is  followed  by  a  rating  session  In 
which  the  expert  makes  Inferences  about  relationships  among  the  objects  and  the 
relatedness  of  the  dimensions.  Then  the  objects  and  dimensions  are  analyzed  to  look 
for  object  clustering  and  for  the  dimensions  used  to  rate  the  Items.  This  Is  done 
based  on  examination  of  the  similarity  of  the  dimensions.  For  the  clustered  objects,  a 
matrix  showing  distances  between  objects  Is  required.  For  the  pairs  of  objects,  the 
absolute  differences  between  the  object  values  are  determined.  For  the  dimensions, 
however,  the  determination  of  distance  factors  Is  not  so  straightforward.  Some  cases 
might  show  correlation  but  show  opposite  values.  A  system  developed  by  Boose  (Reference 
7)  allows  for  examination  of  clustered  dimensions  to  determine  correlation.  Following 
this  analysis,  a  second  Interview  Is  conducted  with  the  expert  to  determine  the  rules 
obtained  from  traits  and  dimensions  found  at  this  phase.  These  rules  are  then  prepared 
for  Implementation  Into  the  prototype  system.  One  advantage  of  this  technique  Is  that 
the  procedures  for  developing  similarity  matrices  are  less  tedious  than  by  directing 
rating  similarity  In  all  pairs.  Another  advantage  Is  that  it  allows  for  the  combination 
of  expertise  from  two  experts  with  different  specialization  within  the  same  field. 
Individual  repertory  grids  can  also  be  used  as  the  basis  for  discussing  and  negoti¬ 
ating  differences  among  the  experts. 


SYSTEM  PROTOTYPING 

It  Is  important  that  the  system  be  prototyped  at  the  earliest  time  possible  so 
that  the  rules  obtained  to  date  can  be  tested  and  modified.  If  necessary.  The  reason 
for  this  Is  that  problems  In  the  expe  -tern  are  not  known  until  actual  implemen¬ 
tation  because  the  system  does  not  ha.-  '"ete  specifications  at  the  start  of  the 
development  effort,  it  should,  however,  eallzed  that  major  rework  might  be  requir¬ 
ed  after  finding  these  problems.  The  prototype  development  efforts  should  concentrate 
on  a  single  representative  problem  at  a  time  and  should  Involve  the  Intended  user  at 
the  earliest  time  possible.  The  advantages  of  doing  this  are  that  (1)  the  user  can 
Immediately  find  any  major  problems  early  In  the  program  and  before  a  considerable 
amount  of  work  has  been  done,  and  (2)  this  also  gives  the  user  an  ability  to  submit 
recommendations  on  how  the  system  design  can  be  made  user-friendly.  The  prototyping 
phase  Is  cyclical  In  that  the  development  effort  pace  varies  between  the  active  sub¬ 
phases  where  new  packages,  tools,  and  Interfaces  are  being  Incorporated  and  those 
times  where  the  knowledge  engineer  Is  going  back  to  the  expert  for  more  knowledge  or 
information.  When  Initially  starting  the  prototype  phase  the  knowledge  engineer  should 
(along  with  support  of  his/her  management  and  the  other  members  of  the  development 
team)  determine  whether  the  implementation  technology  for  the  prototype  will  be  based 
using  an  off-the-shelf  system  shell  or  whether  a  special  purpose  system  will  have  to 
be  built.  The  first  type  offers  the  advantage  of  having  the  shell  Immediately  ready 
foi  the  incorporation  of  rules  as  they  are  developed.  However,  extreme  care  should  be 
taken  by  the  knowledge  engineer  to  insure  that  the  shell  selected  Is  oriented  on  rule 
programming.  The  second  type  offers  the  advantage  of  having  a  tailor-made  system 
which  Is  perhaps  more  conducive  to  producing  the  exact  desired  outputs.  However,  the 
disadvantages  are  that  the  cost  will  be  considerably  higher  for  both  development  and 
maintenance,  It  will  take  more  time  to  develop  and  the  shell  might  not  be  transportable 
to  other  applications  and  expert  system  development  efforts.  Additional  factors  which 
must  be  considered  are  whether  the  tailor-made  shell  development  cost  Include  support 
for  programming  and  consultation  by  the  developer.  When  using  commerlcally  available 
shells,  the  knowledge  engineer  needs  to  ask  questions  about  the  types  of  expert  systems 
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that  have  been  built  before  using  that  shell,  and  the  types  of  support  which  were 
required  and  available  from  the  vendor. 

USER-INTERFACE  DEVELOPMENT 

Once  the  system  has  been  prototyped,  the  first  real  test  of  its  value  is  that  it 
be  user-friendly.  Consequently,  the  knowledge  engineer  must  factor  this  into  the  sys¬ 
tem  design  and  guide  the  development  effort  toward  that  end.  This  process  takes  con¬ 
siderable  time  and  should  therefore  be  a  front-end  development  item.  It  is  Important 
that  the  system  be  simple  enough  to  use  such  that  the  user  does  not  have  to  learn  the 
whole  system.  The  user  should  be  required  to  know  only  the  necessary  inputs  which 
he/she  must  provide  to  get  the  expected  outputs.  These  factors  make  the  system  more 
acceptable  and  of  more  value  to  the  user.  By  the  same  token,  the  system  should  readily 
allow  for  the  knowledge  engineer  to  modify  or  create  new  prototype  packages  with  rel¬ 
ative  ease.  Some  helpful  aids  which  might  be  Incorporated  into  the  prototype  are  goal 
browsers  which  lay  out  the  design  process  as  a  network  of  different  goals  and  displays 
the  status  of  each  goal  when  prompted.  The  browser  allows  for  more  detailed  querying 
of  the  goal  structure  through  menu  interaction  on  the  screen.  It  also  allows  the  user 
to  edit,  undo,  advise,  and  re-execute  goals.  This  feature  will  allow  the  company 
managers  to  know  the  exact  stage  or  phase  at  which  the  effort  is  on  the  milestone 
chart.  This  display  Is  also  updated  by  the  system  Itself  as  the  exploration  of  the 
design  space  continues.  In  addition,  a  separate  interface  design  document  resembling 
the  knowledge  document  should  be  prepared  and  updated  so  that  it  Includes  any  major 
design  decisions  and  analyses  conducted  on  the  interface. 

TESTING  AND  REDEFINITION 

Immediately  after  the  first  version  of  the  prototype  is  developed,  the  system 
should  undergo  Intense  testing.  If  these  developments  consist  of  submodules,  the 
knowledge  engineer,  the  expert,  and  any  other  Intended  users  should  start  testing  the 
system  by  using  test  cases  to  verify  the  rules  contained.  This  process  will  inherently 
generate  new  problems  which  might  give  the  expert  system  development  effort  new  direc¬ 
tion  and  indicate  areas  where  more  work  (i.e.  more  and/or  better  knowledge)  is  requir¬ 
ed.  The  end  result  is  that  a  second  version  prototype  will  usually  be  built  as  new 
knowledge  structure  and  Inference  procedures  are  uncovered.  In  addition,  the  solving 
of  real  problems  causes  a  reimplementation  of  continued  knowledge  programming  or  knowl¬ 
edge  base  maintenance. 

KNOWLEDGE  BASE  MAINTENANCE 

After  the  knowledge  engineer  and  the  prototype  developing  team  have  tested  the 
system,  the  maintaine-s  of  the  system  must  continue  the  process  of  updating  the  system 
as  new  rules/facts  or  .hanges  are  identified.  This  should  be  done  at  all  user  stations 
to  help  calibrate  the  user  Interface  and  extend  the  knowledge  base.  When  this  has  been 
accomplished  at  all  stations,  it  is  then  easier  to  evaluate  the  cost  of  the  resources 
required  as  opposed  to  the  value  of  solving  new  problems. 

SUMMARY 

This  paper  has  presented  some  methodologies  for  acquiring  or  extracting  the  human 
expert  knowledge  via  both  direct  and  indirect  techniques.  Also  included  are  some 
considerations  for  how  to  determine  whether  a  domain  is  suitable  for  an  expert  system, 

what  the  knowledge  engineer  requirements  are,  and  procedures  for  selecting  the  right 

expert  knowledge  via  both  direct  and  Indirect  techniques.  Also  Included  are  some 
considerations  for  how  to  determine  whether  a  domain  is  suitable  for  an  expert  system, 

what  the  knowledge  engineer  requirements  are,  and  procedures  for  selecting  the  right 

expert  or  group  of  experts  for  development  of  an  expert  system. 
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Summary 

VORTEX  is  an  experimental  methodology  for  building  validated  expert  systems.  It  considers  validation 
to  be  an  exercise  in  building  confidence  in  a  system  in  the  users,  procurers,  experts  and  developers.  It 
identifies  the  components  of  validation  concerning  each  participant  and  techniques  for  achieving  validation  and 
embeds  them  in  a  life-cycle  suitable  for  a  novel  technology.  This  paper  presents  the  essential  points  of  the 
methodology  and  some  experiences  from  the  airborne  ASW  application  developed  in  parallel  with  it. 


1.  Introduction 

The  move  towards  more  technologically  complex  mission  systems,  for  coping  with  increasing  mission 
sophistication,  coupled  with  increasing  limits  on  available  manpower  resources  is  leading  to  increased  crew 
workload  and  more  lengthy  training  of  operators  for  many  existing  and  future  aircraft.  RAE  have  identified 
that  expert  systems  technology  may  relieve  this  problem  by  enabling  the  development  of  decision  support  aids 
to  reduce  workload  and  improve  operator  performance.  However,  before  this  technology  can  be  used  opera¬ 
tionally  MOD  need  to  have  confidence  in  its  procurement,  use  and  maintenance.  Validation  is  the  basis  on 
which  confidence  in  a  system  is  built. 

The  validation  of  expert  systems,  though,  remains  a  complex  issue.  Through  VORTEX  (Validation  Of 
Real  Time  EXpert  systems)  RAE  has  developed  a  methodology  that  consists  of  a  description  of  the  develop¬ 
ment  process  from  initial  idea  through  to  operational  use,  and  a  design  approach,  with  tool  support,  that  facili¬ 
tates  this  task.  The  development  process  has  some  common  aspects  with  the  life-cycle  for  any  new  technol¬ 
ogy.  For  example,  it  emphasises  phased  development  and  prototyping  as  a  way  of  establishing  both  require¬ 
ments  and  confidence.  However,  expert  systems  have  their  own  unique  aspects  which  cause  problems  for  vali¬ 
dation,  notably  the  sometimes  individualistic  and  heuristic  knowledge  contained  in  them  and  the  inherent  com¬ 
plexity  of  the  tasks  they  address,  and  are  recognised  to  require  expert  ability  to  solve.  This  means  that  the 
correctness  of  a  generated  solution  is  necessarily  based  on  expert  opinion  and  that  an  initial  specification, 
against  which  to  validate,  is  extremely  difficult  to  produce.  These  expert  system-specific  problems  are  not 
insurmountable  but  do  require  commitment  from  an  expert  to  the  validation  task. 

It  is  not  just  the  expert  who  is  concerned  with  validation.  Issues  such  as  user-friendliness  and  timeliness 
of  response  are  important  concerns  of  the  user,  while  the  developer  assigns  priority  to  maintenance  and  reusa¬ 
bility  and  the  procurer  is  concerned  with  operational  impact.  All  these  aspects  are  encompassed  by  validation 
and  each  should  be  considered  in  a  fully  validated  system. 

However,  there  may  be  some  confusion  about  the  meanings  of  these  terms.  For  instance,  timeliness  could 
be  considered  an  element  of  user-friendliness.  The  ESPRIT  project  “VALID,  Quality  and  Metrics”  [1]  item¬ 
ises  the  criteria  for  quality  software  from  the  developers’  point  of  view..  It  includes  items  such  as  clarity, 
coherence  and  generality,  without  asserting  their  independence.  The  authors  also  describe  components  which 
are  the  quality  factors  from  the  users’  point  of  view,  including  the  items  mentioned  above  such  as  correctness 
and  performance.  The  users’  factors  are  built  up  from  the  developers’  criteria  forming  a  two-tier  model,  indi¬ 
cating  that  a  number  of  criteria  are  related  in  that  they  contribute  to  a  common  user  factor. 

VORTEX  draws  loosely  on  this  model  by  building  another  tier  on  top  of  the  “user  factors”,  where  the 
units  of  the  new  tier  are  the  various  participants  in  an  expert  system  life-cycle,  and  the  links  to  the  (extended) 
set  of  “user  factors”  indicate  their  different  perspectives  on  the  validation  task  as  a  whole.  The  other  main 
influences  on  the  VORTEX  methodology  are  listed  in  the  bibliography. 

In  this  way,  there  is  a  fusion  of  life-cycle,  participants,  validation  components  (such  as  expert  understand¬ 
able  knowledge)  and  techniques  (such  as  expert  understandable  knowledge)  attempting  to  answer  the 
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fundamental  questions  of  when,  who,  what  and  how  to  validate  for  an  expert  system. 

2.  Application  Details 

The  expert  system  methodology  resulting  from  VORTEX  was  based  on  actual  experience  of  building  an 
expert  system  demonstrator.  This  expert  system,  called  the  Sensor  Adviser,  needed  to  be  both  a  plausible 
application  for  improving  mission  performance  and  sufficiently  constrained  for  the  purposes  of  investigating 
and  developing  an  expert  system  methodology.  It  was  originally  intended  to  consider  Anti-Submarine  War¬ 
fare  (ASW)  data  fusion  but  after  applying  the  techniques  described  in  the  initiation  phase  the  choice  moved  to 
a  decision  aid  for  Merlin  Observers  for  sensor  selection  and  deployment  in  ASW. 

Sensor  selection  chooses  the  best  sensors  from  the  Merlin  sensor  suite  to  deploy  in  a  particular  tactical 
situation,  whilst  sensor  deployment  chooses  the  best  way  to  use  the  sensor.  Merlin  is  the  name  given  to 
EH101  helicopters  planned  as  the  aircraft  for  Invincible-class  ships.  It  has  a  long  range  and  good  endurance, 
and  is  equipped  with  sonobuoys,  active-dipping  sonar  (ADS),  radar  and  magnetic  anomaly  detector  (MAD). 
Operating  autonomously  in  its  ASW  role,  the  crew  consists  of  a  pilot  in  the  forward  cockpit,  and  an  Observer 
and  an  acoustics  system  operator  in  the  rear  cockpit. 

Observers  are  likely  to  have  spent  two  years  in  the  Navy  either  airborne  or  with  some  command  responsi¬ 
bility  on-board  a  ship,  followed  by  18  months  training  in  ASW  tactics,  followed  by  three  to  four  years  of 
practical  experience.  They  are  responsible  for  the  mission,  but  need  to  negotiate  with  the  pilot  over  tactics 
and  risks,  particularly  where  these  represent  a  threat  to  the  aircraft.  The  Demonstrator  aims  to  increase  the 
effectiveness  of  Merlin  Observers  on  sorties.  It  demonstrates  how  its  operational  equivalent  could  substan¬ 
tially  assist  Observers  in  achieving  their  mission  objectives,  by  improving  performance  and  smoothing  work¬ 
load. 

The  Demonstrator  improves  performance  by  helping  an  Observer  to: 

•  consider  all  relevant  factors  in  the  selection  and  deployment  of  available  sensors 

•  avoid  narrow-thinking,  and  so  reduce  the  risk  of  making  poor  decisions 

•  consider  overlooked  options  at  times  when  it  is  not  clear  what  subsequent  actions  to  take,  such  as  losing 

contact  with  a  submarine  that  was  previously  being  tracked 

An  Observer’s  workload  can  vary  considerably  during  a  sortie.  For  example,  workload  is  often  low  when 
flying  to  a  barrier  and  excessive  when  prosecuting  a  target.  The  Demonstrator  helps  to  even  out  these  wide 
variations.  When  workload  is  low  the  Observer  can  use  the  Demonstrator  to  explore  alternative  tactics  and 
prepare  for  the  mission.  When  workload  is  high  the  Observer  can  use  the  Demonstrator  for  advice  on  sensor 
selection  and  deployment. 

3.  The  Sensor  Adviser 

The  VORTEX  Demonstrator  is  written  in  Common  Lisp  with  Flavours  to  run  on  a  Symbolics  3650 
machine.  It  consists  of  a  mouse  driven  graphical  interface,  a  set  of  validation  tools,  an  inference  engine,  a 
Knowledge  Base  Manager  (KBM)  and  the  KB  itself.  These  are  described  below  in  sufficient  detail  to  under¬ 
stand  illustrative  examples  which  will  be  raised  during  presentation  of  the  methodology.  Figure  1  gives  a 
block  diagram  of  the  relationship  of  the  various  parts. 

3.1.  Graphical  Interface 

In  order  to  aid  both  user  and  expert  understanding  of  the  proposed  system,  the  interface  has  been 
designed  to  simulate  that  which  will  be  available  to  Merlin  Observers.  One  window  simulates  Merlin’s  Com¬ 
mon  Control  link  (CCU  )  using  mouse  sensitive  buttons.  The  Observer  interacts  with  the  system  through  the 
CCU,  navigating  short  menu  paths  which  may  require  input  from  the  CCU  keypad.  The  alert  key  allows  the 
Observer  to  interrupt  the  Adviser  and  provide  unsolicited  information  which  can  chanpe  the  current  line  of 
reasoning.  The  Sensor  Adviser  outputs  its  advice  on  a  simulation  of  the  Common  Display  Unit  (CDU), 
together  with  a  concise  explanation  of  the  rationale  for  this  advice.  It  is  intended  that  this  output  would  be 
imposed  on  Merlin’s  tactical  plot. 

Additional  windows  are  used  for  demonstration  and  development  purposes.  One  window  provides  a 
detailed  trace  of  the  reasoning  being  undertaken  by  the  Sensor  Adviser’s  inference  engine.  Another  controls 
the  scenario  and  provides  access  to  both  the  internal  state  of  the  Sensor  Adviser  and  to  a  suite  of  validation 
tools.  It  enables  system  level  interrupts  to  allow  user  input  of  information  which  would  normally  be  available 
from  Merlin’s  mission  system,  such  as  sonobuoy  detections.  These  windows  would  not  be  present  on  board 
Merlin  but  might  exist  in  an  operational  support  station.  Their  purpose  is  to  build  the  confidence  experts  and 
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developers  have  in  the  Sensor  Adviser  and,  therefore,  they  form  part  of  the  validation  tool  support. 

3.2.  Knowledge  Base 

It  is  important  that  the  knowledge  incorporated  in  an  expert  system  be  easily  understood  by  experts  if 
they  are  to  fulfil  their  validation  responsibilities.  Although  individual  production  rules  may  be  understood, 
their  interaction  is  much  harder  to  grasp.  Moreover,  implementational  constraints  may  compound  this  problem 

The  Sensor  Adviser  knowledge  base  consists  of  two  kinds  of  knowledge: 

•  domain  knowledge  such  as  threat,  sensor  and  weapon  data 

•  problem-solving  knowledge,  ASW  tactics  and  their  application 

Domain  knowledge  is  organised  in  the  familiar  object  hierarchy  with  inheritance,  so  that  a  MAD  is  a  type  of 
sensor,  which  is  a  type  of  object,  and  so  on.  Problem  solving  knowledge  is  organised  in  a  procedural  form 
which  reflects  the  way  experts  decide  on  the  best  tactics  for  a  given  scenario.  Tasks  are  therefore  linked  with 
key  decisions  made  by  experts.  For  example  the  task  go-passive  is  linked  to  the  decision  variable  active- 
versus-passive.  Tasks  may  depend  on  sub-tasks  which  are  necessary  in  order  to  perform  the  parent  task.  For 
example,  go-passive  is  a  sub-task  of  decide-active-versus-passive.  The  latter  will  reason  about  such  things  as 
water  conditions  while  the  former  will  actually  update  the  decision  variable.  Tasks  are  also  represented  as 
objects  in  the  object  hierarchy. 

The  internal  reasoning  within  a  procedural  task  is  actually  implemented  using  declarative  production  rule ,. 
The  above  problem  with  understanding  the  interactions  of  rules  is  removed  since  they  are  all  constrained  to 
work  on  one  task,  and  it  is  the  interaction  of  the  tasks  which  trace  the  flow  of  reasoning,  and  this  is  much 
more  natural  for  the  expert.  In  this  way  knowledge  base  modularity  is  achieved  and  the  natural  expressive¬ 
ness  of  production  mlcs  is  retained  without  the  associated  disadvantage  of  complexity  of  interaction.  Of 
course,  from  a  wider  perspective,  different  tasks  could  have  different  problem  solving  methods  within  them, 
including  algorithmic  techniques,  but  validation  requirements  have  prompted  the  use  of  expert  understandable 
production  rules. 
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Figure  1 :  Sensor  Adviser  Architecture 
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3.3.  Inference  Engine 

The  inference  engine  is  the  piece  of  software  which  systematically  applies  the  knowledge  in  the  Sensor 
Adviser.  It  is  therefore  conventional  software  and  is  suitable  for  verification  and  validation  using  conventional 
techniques.  VORTEX  concentrates  on  that  part  of  the  validation  process  which  is  special  to  expert  systems 
and  so  does  not  consider  validation  of  inference  engine  code. 

The  operation  of  the  inference  engine  is  dependent  on  the  knowledge  representation  so  a  brief  description 
of  its  operation  is  given  here  for  completeness  and  to  aid  understanding  of  the  knowledge  representation. 

The  inference  engine  maintains  a  schedule  of  tasks  for  processing  in  order.  These  tasks  may  schedule 
sub-tasks,  which  are  inserted  in  the  schedule  before  the  task  following  the  parent  task.  Completed  tasks  are 
not  removed  from  the  schedule,  but  remain  as  a  record  of  the  reasoning  process.  This  record  is  necessary  not 
only  for  validation  puiposes,  but  during  execution  when  interrupts  may  arrive  from  the  user,  in  the  form  of 
unsolicited  information  or  from  the  (simulated)  mission  system.  These  interrupts  may  invalidate  previous  rea¬ 
soning,  eg  a  change  of  expected  threat  type.  This  will  lead  to  the  slams  of  some  decision  variable  moving 
from  “decided”  to  “undecided”.  Since  tasks  are  connected  to  decision  variables,  reasoning  will  need  to  be 
backtracked  to  the  >st  point  before  the  modified  decision  variable  was  used,  by  truncating  the  schedule  at  the 
appropriate  task.  This  then  becomes  the  current  task  and  reasoning  resumes. 

3.4.  Knowledge  Base  Manager 

Any  changes  to  the  knowledge  base  required  during  reasoning  are  performed  through  the  Knowledge 
Base  Mangaer  (KBM).  It  performs  limited  verification  by  the  use  of  “strong  typing”.  For  example,  a  vari¬ 
able  such  as  sea-state  is  a  numerical  variable  with  cardinality  one  and  range  0-9.  If  this  is  updated  during  a 
mission,  the  KBM  will  prevent  anything  other  than  a  single  number  in  the  specified  range  becoming  the  new 
value.  Strong  typing  also  applies  to  all  objects  and  their  attributes  in  the  KB.  Note  that  the  constraints  do  not 
have  to  be  language  primitives,  e.g.  the  variable  recording  expected  threat  must  have  a  value  which  is  actually 
some  submarine  from  the  domain  knowledge. 

The  KBM  also  perfonns  the  appropriate  housekeeping  functions  when  backtracking  of  reasoning  occurs. 
The  inference  engine  only  records  the  schedule  of  tasks  and  schedules  new  tasks.  It  is  the  KBM  which  deter¬ 
mines  the  effects  of  external  interrupts,  deciding  which  decision  variables  are  affected  and  how  far  back  to 
undo  reasoning.  The  KBM  operates  both  at  run-time  and  compile-time,  so  it  also  provides  verification  support 
when  developing  and  maintaining  the  KB.  Strong  typing  and  tracing  tasks  which  depend  upon  a  modified 
object  detects  a  very  large  number  of  problems  which  would  otherwise  be  extremely  expensive  to  track  down. 

3.5.  Tool  Support 

Experience  and  common  sense  suggest  that  effective  validation  requires  tool  support.  The  VORTF' 
ject  has,  therefore,  developed  a  number  of  tools  dealing  with  the  validation  and  verification  issues  sp-  j 
expert  systems.  The  KBM  described  above  includes  verification  tools.  The  trace  and  explanation  fuvuii.es 
mentioned  in  the  graphical  interface  section  are  validation  tools.  A  number  of  other  tools  bridge  the  gap 
between  assessing  specific  advice  and  inspecting  the  incorporated  knowledge.  These  are  the  Task-Task 
Identifier,  Task-Object  Identifier,  Decision-Task  Identifier,  Object-Task  Identifier  and  Task-Circularity 
Identifier.  The  first  four  produce  graphical  displays  indicating  a  specific  item  and  the  items  connected  with  it. 
For  example,  the  Task-Object  Identifier  displays  all  objects  read  from  and  written  to  by  a  particular  task.  More 
details  direct  from  the  KB  (and  thus  in  expert  readable  form)  are  available  by  mouse  selection  of  any  of  the 
objects  displayed.  The  Task-Circularity  Identifier  locates  any  potential  problems  caused  b;  a  Task  that  could 
schedule  sub-tasks  and  then  itself  be  scheduled  by  the  sub-tasks  or  their  descendants.  The  circularities  might 
not  occur  at  run-time  due  to  the  run-time  conditions  not  causing  that  particular  reasoning  path  to  be  followed. 
However,  it  is  necessary  to  know  where  they  might  occur  to  validate  the  KB. 

4.  Validation  and  Verification  Techniques 

There  are  a  number  of  different  validation  and  verification  techniques  which  address  the  special  issues  of 
expert  systems: 

Application  assessment  -  we  liave  identified  a  number  of  criteria  for  assessing  the  suitability  ot  expert-system 
technology 

the  existence  of  human  experts 

the  demand  for  expertise  is  too  great  for  the  available  supply 

there  is  a  need  to  improve  the  consistency  of  operational  decision  making 

there  is  current  planning  for  future  unavailability  of  expertise 


27-5 


the  decision  making  quality  is  key  to  a  task 

the  area  of  the  task  requiring  expertise  is  well  defined  and  can  be  packaged  practically 

the  user  can  share  in  the  decision  making  process 

the  amount  of  user  input  and  output  is  not  onerous 

there  is  an  existing  or  potential  mechanism  for  maintaining  ‘best  practice’ 

the  users’  trust  can  in  principle  be  established 

the  envisaged  system  supports  rather  than  replaces  the  user’s  decision  making  process 

the  reasoning  experts  employ  is  explicit  rather  then  perceptual 

there  is  access  to  experts  who  either  do  the  job  or  have  done  it  recently 

uncertainties  in  the  problem-solving  process  are  handled  in  a  well-defined  and  meaningful  way 

the  necessary  ‘input’  and  ‘output’  to  the  problem-solving  process  is  well  understood 

the  “radio-link”  test  works  [this  test  requires  the  expert  to  enquire  about  situations  as  if  over  a  radio;  the 

development  team  then  suggest  responses  for  criticism  and  are  thus  able  to  establish  an  understanding  of 

the  decisions  made  and  the  factors  required  to  make  them] 

the  vocabulary  and  form  of  the  knowledge  statements  that  comprise  the  knowledge  base  can  be  made 
familiar  to  the  expert  and  to  the  users 

the  development  team  is  experienced  in  expert-system  development 

early  review  and  prototyping  points  are  established,  and  feedback  from  experts  and  users  is  encouraged 
that  real-time  and  spatial  considerations  in  the  problem  are  handled  in  a  way  which  is  understood 
Test  cases  -  These  are  by  far  the  most  useful  validation  techniques  that  we  have  used  in  VORTEX.  Different 
test  cases  can  be  used  for  different  validation  elements.  However,  a  single  test  case  gives  only  partial 
confidence  in  the  incorporated  expertise.  Typically  expert  systems  have  an  extremely  large  number  of 
equivalence  classes,  and  so  exhaustive  testing  is  inappropriate. 

Panel  of  taste  -  This  is  a  mechanism  favoured  by  those  participating  in  our  validation  experiments  for  estab¬ 
lishing  the  “right”  answers  to  questions  uncovered  with  test  cases  is  to  establish  an  authorized  “panel  of 
taste”  to  arbitrate  on  what  knowledge  is  correct. 

Subjective  judgement  -  many  of  the  aspects  of  developing  expert  systems  discussed  elsewhere  in  this  report 
conspire  to  make  subjective  judgement  much  more  important  than  is  apparently  the  case  in  more  conventional 
technologies. 

Asking  the  right  questions  at  the  right  time  -  the  weaknesses  of  a  subjective  judgement  approach  can  be 
significantly  ameliorated  if  the  right  questions  are  asked  at  the  right  time.  This  gives  the  maximum  chance  of 
detecting  problems  early,  when  they  are  much  cheaper  to  fix.  Our  results-oriented  life-cycle  and  validation 
approach  are  designed  to  achieve  this  end. 

Tool  Support  -  Developing  and  employing  V  &  V  tools  is  an  important  technique:  tools  fall  into  a  number  of 
different  categories: 

•  Type-checking  verification  tools,  which  can  operate  at  both  compile  time  and  run  time.  These  catch 
many  “clerical”  mistakes,  which  would  otherwise  be  very  expensive  to  track  down. 

•  Structured  editors,  which  ensure  that  only  syntactically  acceptable  knowledge  statements  can  be  entered 
into  the  system.  Again,  by  catching  such  errors  at  the  syntactic  level,  much  effort  can  be  avoided. 

•  Completeness  tools,  which  verify  that  no  referenced  items  of  knowledge  are  duplicated  or  omitted. 

•  Explanation  tools,  which  build  confidence  in  users  that  the  answers  will  meet  the  required  level  of  com¬ 
petence. 

•  Trace  tools,  which  build  confidence  in  experts  that  the  knowledge  is  correct,  in  that  correct  answers  are 
being  generated  by  appropriate  lines  of  reasoning. 

•  Inspection  tools,  which  build  confidence  that  the  knowledge  is  both  correct  and  complete,  by  allowing 
the  expert  to  inspect  the  knowledge  base  flexibly,  and  follow  relationships  between  knowledge  fragments. 

•  Consistency  tools,  which  assist  validation,  for  example  by  ensuring  that  all  dependant  elements  of 
problem-solving  knowledge  are  re-examined  when  a  statement  of  domain  knowledge  is  modified. 

In  the  VORTEX  project,  versions  of  these  tools  have  been  incorporated  in  the  KBM. 

5.  Life-Cycle 

This  section  describes  the  VORTEX  methodology  in  detail.  It  presents  the  Initiation,  Feasibility,  Develop¬ 
ment,  Delivery  and  Operation  phases,  identifying  the  outputs  and  the  most  important  validation  components  of 
each  phase.  For  each  component,  the  appropriate  participants  and  their  perspectives  are  identified  and  an  indi¬ 
cation  of  the  techniques  employed  to  validate  the  components  is  given. 


5.1.  Initiation  Phase 

The  first  stage  is  the  idea  that  some  aspect  of  a  requirement  may  be  met  by  an  expert  system,  either 
through  apparent  relevance  of  the  technology  or  through  belief  that  conventional  systems  cannot  do  the  job. 
Neither  is  a  guarantee  of  applicability  or  success.  The  assessment  indicators  described  above  may  be  used  for 
this  purpose. 

An  initial  specification  of  all  aspects  of  the  expert  system  should  be  produced.  It  provides  the  first  oppor¬ 
tunity  for  validation  but  it  must  be  accepted  that  for  expert  systems  the  specification  will  evolve  during  the  life 
cycle  because  the  technology  is  new,  so  procurers  cannot  understand  it  well,  and  because  the  nature  of  exp.' 
tise  is  such  that  it  is  difficult  to  specify  and  hard  to  manipulate  by  analogy.  For  expert  systems  in  general, 
testing  the  system  is  a  fundamental  part  of  elicitation,  and  hence  development,  and  so  leads  to  modifications 
of  the  KB,  which  has  an  effect  on  the  specification.  As  a  case  in  point,  the  Sensor  Adviser  was  originally 
only  intended  to  make  the  decision  to  go  active  or  stay  passive.  After  a  number  of  elicitation  sessions  it 
became  apparent  that  both  sensor  selection  and  deployment  advice  should  be  provided. 

The  procurer  is  interested  in  the  impact  of  the  system,  ie  the  observable  benefit.  This  requires  the  produc¬ 
tion  of  a  statement  of  impact  which  should  address  the  following  questions: 

•  What  externally  observable  measure  of  performance  will  be  improved  through  deploying  the  expert  sys¬ 
tem?  For  the  Sensor  Adviser,  examples  include  reducing  time  to  weapon  in  the  water  or  reducing  the 
number  of  failed  prosecutions. 

•  Who  will  achieve  the  improved  measure  of  performance  by  using  the  expert  system?  E.g.  novice,  aver¬ 
age,  expert  and  fatigued  crews. 

•  When  will  the  improved  measure  of  performance  be  achieved?  E.g.  against  modem,  unco-operative, 
multiple  and  expert  threats. 

A  key  insight  from  the  VORTEX  validation  experiments  is  that  expert  system  decision  support  tools  may 
often  yield  benefit  from  catching  mistakes  and  preventing  errors  rather  than  from  systematically  out¬ 
performing  crews.  This  should  influence  the  way  that  the  statement  of  impact  expresses  the  observable 
benefits. 

The  validation  components  associated  with  this  phase  are  described  below,  together  with  those  concerned 
with  them.  The  output  of  this  phase  is  paper  based  and  so  the  only  technique  that  can  be  used  is  eliciting 
subjecti-e  opinion  on  the  proposals  made. 

Impact 

The  statement  of  impact  is  usually  elicited  by  developers  from  procurers,  perhaps  with  the  assistance  of 
experts.  Experienced  developers  should  seek  to  reduce  over-ambitious  requirements,  but  all  concerned 
should  realise  this  will  probably  only  be  a  target  statement. 

Application  assessment 

The  developers  apply  the  indicators  of  section  four  to  initial  discussions  with  experts.  For  the  Sensor 
Adviser  the  radio  link  test  was  found  to  be  particularly  useful.  For  instance,  two  initially  suggested  appli¬ 
cations  were  improving  tactical  display  quality  to  show  only  “interesting  and  relevant”  information,  and 
automatic  alert  of  mission  critical  situations  which  require  an  Observer’s  immediate  attention.  Both  were 
eliminated  since  they  are  highly  perceptual  tasks  that  are  difficult  to  analyse  in  the  manner  required  for 
the  radio-link  test.  Expert  systems  cannot  model  perceptual  tasks.  On  the  other  hand,  the  spatial  infor¬ 
mation  required  for  the  Sensor  Adviser  which  might  be  considered  perceptual  in  nature  can  be  verbalised 
in  terms  of  courses  of  threats  and  standard  patterns  of  buoy  development 

Knowledge  Availability 

This  can  be  a  problem  when  the  procurer  views  expert  system  techniques  as  a  panacea.  Developers  must 
have  access  to  knowledge  to  build  the  KB.  Domain  knowledge  such  as  ship  characteristics  will  often  be 
available  in  reports  and  manuals.  Problem  solving  knowledge  may  be  available  from  tactics  manuals  but 
expert  level  knowledge  will  rarely  be  documented.  Commitment  from  experts  is  therefore  vital  for  suc¬ 
cess. 

Functionality 

The  user  needs  to  be  satisfied  that  the  system  will  provide  all  functions  required.  Functionality  is  con¬ 
sidered  to  include  capabilities,  explanations,  changes  of  mind,  division  of  labour  and  what-if  questions. 
During  this  phase,  the  developer  seeks  confirmation  of  valuable  and  workable  functionality.  For  the 
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Sensor  Adviser,  experts  were  consulted  since  Merlin  is  a  future  system  and  there  are  no  users  currently. 
Integration 

The  plausibility  of  integrating  an  expert  system  with  the  anticipated  platform  requires  the  involvement  of 
expert  system  developers  and  those  who  maintain  the  platform. 

5.2.  Feasibility  Phase 

The  output  of  this  phase  is  a  feasibility  prototype  and  a. feasibility  report.  Since  tangible  software  is 
being  produced,  the  subjective  speculations  of  the  initiation  phase  become  mostly-subjective  evaluations  of  a 
real  artifact  This  is  a  major  improvement  It  allows  use  of  the  techniques  of  inspection  and  test  cases  and  is 
important  for  developing  the  knowledge  representation,  which  has  been  identified  as  a  key  element  in  valida¬ 
tion.  The  original  proposal  for  VORTEX  omitted  the  feasibility  prototype  on  the  grounds  of  cost  but  one  was 
built  since  it  was  discovered  to  be  so  important 

However,  it  is  important  to  remember  that  the  feasibility  prototype  is  just  a  prototype.  As  such  it  is  a  tool 
for  conducting  experiments  that  will  lead  to  a  final  system.  It  is  not  the  final  system  itself,  even  though  ele¬ 
ments  of  it  may  be  retained.  Prototyping  is  an  experimental  activity  and,  like  all  experiments,  should  be  well 
planned  before  commencing,  with  an  expected  use  for  the  results.  In  the  expert  system  context,  the  results  are 
typically  used  to  improve  knowledge  representation  and  inference,  with  the  goal  of  obtaining  better  solutions 
for  the  current  test  cases.  Planning  prevents  prototyping  becoming  hacking. 

The  validation  components  associated  with  this  phase  are  described  below. 

Confidence 

Experts  want  to  know  their  expertise  can  be  captured,  procurers  that  the  statement  of  impact  can  be  met, 
and  developers  that  the  technical  approach  is  appropriate.  Confidence  is  a  subjective  matter.  The  feasibil¬ 
ity  prototype  provides  procurers  with  a  demonstration  of  the  feasibility  of  expert  system  technology 
applied  to  the  application,  experts  with  advice  from  test  cases  and  an  expression  of  their  expertise,  and 
developers  with  feedback  from  the  participants  involved  in  the  development  process.  There  is,  therefore, 
tangible  evidence  to  aid  confidence  building. 

Cost 

Developers  will  be  able  to  revise  cost  estimates  from  building  the  prototype.  This  leads  to  a  decision 
point  for  the  procurers  about  whether  the  anticipated  benefits  warrant  the  cost. 

Competence 

At  this  stage  it  is  a  matter  of  expert  opinion  as  to  whether  the  competence  displayed  by  the  limited  feasi¬ 
bility  prototype  is  likely  to  support  the  anticipated  operational  impact.  This  opinion  should  be  explicitly 
sought  and  recorded. 

Human  Computer  Interaction 

If  the  prototype  environment  is  sufficiently  realistic,  the  users  should  evaluate  the  proposed  HCI.  Other¬ 
wise,  the  judgement  is  made  on  the  basis  of  general  HCI  considerations,  such  as  Schneidermann’s  work 
on  direct  manipulation,  and  on  expert  opinion  about  the  operational  task.  With  the  Sensor  Adviser,  the 
CCU  and  CDU  proposed  for  Merlin  were  emulated  as  described  above.  This  meant  that  users  could 
evaluate  the  HCI  (although  for  Merlin  the  experts  are  in  effect  the  only  users).  Initially,  the  system  was 
designed  to  ask  for  any  missing  infoimation  when  advice  was  requested.  This  proved  unpopular  so  the 
format  was  changed  to  provide  advice  using  defaults,  while  making  observations  that  information  was 
missing  and  allowing  the  user  to  provide  it  as  unsolicited  information. 

Timeliness 

The  user  is  concerned  that  advice  is  generated  at  the  right  time.  As  well  as  speed  of  response,  appropri¬ 
ateness  of  advice  is  important.  By  this  stage,  the  analysis  of  the  application  should  have  identified  any 
areas  of  time  criticality.  In  tactical  decision  aids,  these  are  likely  to  be  relatively  well  contained,  as  was 
the  case  with  the  Sensor  Adviser,  but  they  should  be  addressed  in  the  prototype.  Selectiveness  of  advice 
is  a  problem  for  knowledge  elicitation.  In  fact,  VORTEX  considers  solutions  to  response  time  problems 
to  be  a  knowledge  elicitation  task  also.  It  makes  sense  to  determine  how  the  expert  responds  to  time  pres¬ 
sure  and  to  emulate  these  methods,  rather  than  to  attempt  to  deal  with  the  problem  in  some  automatic 
manner.  The  task  structure  described  above  allows  different  problem  solving  methods  to  be  used  as 
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required. 

Scope 

The  user  is  concerned  with  the  breadth  of  applicability  of  the  expert  system.  This  will  be  expressed  in 
teims  of  scenarios  that  can  be  addressed.  The  scenarios  will  foim  the  basis  of  the  test  cases  and  are  there¬ 
fore  selected  by  the  expert  to  cover  the  different  categories  that  must  be  addressed  and  the  differing  com¬ 
plexities.  Scenarios  which  do  not  need  to  be  considered  may  also  be  specified.  The  user  then  needs  to 
know  the  range  of  scenarios,  how  competence  varies  over  the  range  and  how  performance  degrades  at  the 
boundaries.  The  feasibility  prototype  only  deals  with  some  of  the  test  cases  but  the  results  of  these 
s.tould  be  used  to  provide  answers  to  the  users’  requirements.  In  terms  of  ASW,  scenarios  will  be 
specified  by  a  particular  threat  in  a  particular  theatre  of  operations,  such  as  the  North  Atlantic,  with  details 
of  weather  and  threat  behaviour  and  so  on.  The  scope  at  this  stage  also  needs  to  be  validated  by  both 
experts  and  users  against  the  anticipated  impact. 

Knowledge 

At  this  stage  the  expert  is  concerned  with  whether  the  knowledge  required  to  achieve  the  anticipated 
impact  can  be  incorporated,  rather  than  trying  to  validate  the  limited  knowledge  in  the  prototype.  The 
developers  are  concerned  that  the  representation  and  inference  facilities  are  adequate  for  the  final  system. 
The  former  decision  comes  from  the  test  case  results  and  the  latter  from  feedback  from  the  expert  on 
these  results.  For  the  Sensor  Adviser,  the  inclusion  of  decision  variables  (see  above)  aided  expert  under¬ 
standing  and  allowed  dependancy-directed  backtracking  of  reasoning  to  be  conveniently  implemented. 
These  were  missing  in  the  prototype,  which  highlighted  the  problem. 

Integration 

Having  derived  an  integration  strategy,  the  developers  must  validate  its  feasibility  within  the  context  of 
the  known  requirements  for  the  expert  system  module  for  the  target  operational  platform.  The  Sensor 
Adviser  is  written  in  Common  Lisp.  This  would  be  easier  to  translate  to  Ada  for  military  operational  use 
than  if  it  had  been  developed  using  a  toolkit  such  as  KEE  or  ART. 

5.3.  Development  Phase 

Development  of  the  expert  system  can  be  divided  into  a  conventional  part,  the  infra-structure  which 
includes  the  inference  engine,  and  an  expert-specific  part,  the  knowledge  representation  and  the  knowledge 
base.  The  infra-structure  is  highly  dependant  on  the  knowledge  representation  chosen  and  configuration 
management  of  the  two  is  a  practical  issue  of  some  importance.  However,  it  is  a  conventional  project 
management  problem  and  so  is  not  covered  here.  The  Sensor  Adviser  infra-structure  provides  the  interface, 
KBM,  Inference  Engine  and  validation  tools  and  is  described  in  section  three. 

Development  of  the  KB  continues  throughout  the  life-cycle.  During  delivery,  (which  covers  extensive 
user  trials)  the  close  relationship  between  testing  and  acquisition  will  inevitably  spawn  new  knowledge  to  be 
included  as  part  of  the  maintenance  phase.  As  tactics  change  with  improvements  in  sensor  capability  or  threat 
characteristics  and  behaviour,  so  must  the  knowledge  base  be  maintained.  Disposal  of  the  system  occurs  when 
the  benefit  it  delivers  outweighs  the  cost  of  its  maintenance.  A  common  mistake  during  initial  development  is 
to  perform  too  much  domain  modelling  with  no  clear  idea  about  how  the  problem-solving  knowledge  will  util¬ 
ise  it.  Usually  it  is  better  to  be  driven  by  the  problem-solving  knowledge  and  acquire  domain  knowledge  as 
required.  This  is  based  on  the  assumption  that  for  validatable  decision  support  systems  it  is  better  to  adopt 
the  procedural  approach  to  knowledge  representation  discussed  in  section  three. 

In  acquiring  knowledge,  the  following  techniques  have  been  found  to  be  most  useful: 

•  The  radio-link  test  -  good  at  getting  experts  to  articulate  their  problem-solving  approach  and  determining 
appropriate  interaction  with  system. 

•  Knowledge  animation  -  illustrating  the  consequences  of  knowledge  statements  aids  the  expert  in  spotting 
shortcomings  and  suggesting  extensions  and  qualifications. 

•  Constructed  test  cases  -  varying  test  cases  to  identify  boundary  conditions  between  different  solutions 
and  methods. 

Inspection  of  the  knowledge  base  is  a  further  potential  means  of  knowledge  acquisition,  but  is  time  intensive 
on  the  expert  and  so  has  not  been  extensively  used  by  the  VORTEX  project 

The  output  of  this  phase  is  the  final  system  for  delivery.  The  components  of  validation  associated  with 
the  development  phase  are  described  below. 
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Impact 

By  the  end  of  the  development  phase,  the  validation  of  the  target  statement  of  impact  will  be  to  a  much 
greater  degree  of  confidence  because,  while  still  dependant  on  the  judgement  of  the  expert,  the  procurers 
will  be  able  to  see  the  results  for  a  full  range  of  example  scenarios. 

Confidence 

All  the  critical  expert  system  issues  will  be  resolved  so  confidence  of  procurers  and  experts  will  be  high. 
The  remaining  areas  of  uncertainty  concern  integration  and  the  accuracy  of  projected  operational  condi¬ 
tions  compared  to  the  real  world. 

Cost 

The  technology  related  cost  uncertainties  will  be  largely  resolved  and  so  validation  of  cost  will  be  reli¬ 
able.  The  procurers  carry  out  this  validation  with  information  from  developers. 

Competence 

The  expert  system  now  embodies  the  competence  statement.  The  advice  generated  for  all  specified 
scenarios  defines  the  competence  achieved,  which  the  user  validates  (drawing  on  expert  opinion  and  panel 
of  taste)  against  the  target  competence. 

HCI 

The  HCI  is  fully  defined  and  should  be  validated  by  users  against  the  operational  environment  in  which  it 
will  operate. 

Timeliness 

Tht,  full  range  of  scenarios  are  available  for  selection  of  time  critical  test  cases  by  the  expert  for  valida¬ 
tion  by  the  user,  using  developers’  projections  of  relative  performance  in  the  operational  environment. 


Functionality 

Now  fully  defined  to  be  validated  by  the  user  against  the  specification. 

Scope 

All  test  scenarios  can  be  validated  against  the  specification  by  users.  However,  the  users  and  expert  must 
also  consider  the  flexibility  to  deal  with  future  developments  in  the  problem  domain. 

Knowledge 

By  the  end  of  development,  validation  by  the  experts  of  the  incorporated  knowledge  will  depend  upon  a 
number  of  factors: 

•  verification,  primarily  through  the  use  of  tools,  of  typing  (compile  and  run  time),  syntax  and  com¬ 
pleteness  of  the  knowledge  base 

•  inspection  of  KB  listings,  both  by  experts  involved  in  the  development  and  by  other  experts 

•  assessment  of  the  advice  generated  in  scenarios  selected  by  experts  to  probe  completeness  and 
correctness  of  the  knowledge 

•  browsing,  using  tools,  of  knowledge  related  to  issues  arising  from  and  suggested  by  the  test  cases. 
The  developers  will  now  have  full  confidence  that  appropriate  facilities  exist  for  the  initial  KB  release. 
They  must  also  validate  the  appropriateness  of  the  facilities  against  an  assessment  of  future  needs  as 
external  requirements  evolve.  Validation  of  the  software  constituting  the  knowledge  representation  and 
inference  facilities  is  addressed  by  the  developers  in  a  conventional  manner. 

Integration 

At  this  stage  a  detailed  integration  plan  will  exist  and  the  developers  will  validate  this  3gainst  the  require¬ 
ment. 

In  practice,  military  systems  progress  through  the  development  stage  using  increasingly  realistic  world 
models  which  include  paper  studies,  laboratory  demonstators,  simulators  and  experimental  aircraft.  This  pro¬ 
gression  increases  the  complexity  and  scope  of  the  validation  process,  but  to  compensate  for  this  there  are 
fewer  artificial  barriers  distracting  the  the  expert  from  the  knowledge  and  its  validation.  The  specification  and 
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confidence  building  also  develop  in  parallel  with  this  progression. 

VORTEX  is  currently  at  the  laboratory  demonstator  stage.  A  number  of  validation  experiments  have 
been  carried  out  using  four  experts  other  than  the  main  expert  used  for  most  of  the  project,  each  with  slightly 
different  orientations  on  the  ASW  domain,  e.g.  training  as  opposed  to  operation.  The  experts  followed  the 
same  set  of  scenarios,  and  were  asked  to  say  what  they  would  do  at  each  stage,  just  before  the  Sensor  Adviser 
gave  its  advice.  If  there  were  discrepancies  the  experts  were  asked  to  make  their  subsequent  decisions  assum¬ 
ing  that  they  had  followed  the  Sensor  Adviser’s  advice,  to  ensure  consistency  over  the  experiments.  The 
discrepancies  and  the  reasoning  behind  them  were  recorded.  This  meant  that  knowledge  was  elicited,  indicat¬ 
ing  the  close  relationship  between  testing  and  elicitation. 

These  experiments  contributed  to  confidence  building,  competence  assessment,  scope  and  HCI  component 
validation,  by  identifying  areas  for  further  development  The  knowledge  representation,  timeliness  and  func¬ 
tionality  component  reqirements  were  demonstrated  to  be  satisfied  for  the  current  scope.  Future  work  is 
intended  to  address  the  knowledge  elicited  during  the  experiments.  This  will  expand  the  scope  to  deal  with 
more  realistic  scenarios.  The  HCI  will  also  be  developed  to  produce  a  graphical  display  of  buoy  deployment 
as  all  experts  found  it  difficult  to  keep  track  of  the  textual  advice  currently  provided.  The  work  is  intended  to 
remain  as  a  laboratory  demonstrator,  but  once  the  new  scope  has  been  validated  and  there  is  sufficient  com¬ 
petence  across  it,  there  should  be  enough  confidence  to  move  to  an  integation  task  with  a  simulator. 

5.4.  Delivery  and  Operation  Phases 

The  major  activity  of  delivery  is  integration  with  the  platform  hardware  and  software,  a  conventional 
engineering  exercise,  and  validation  of  the  system  using  the  real  environment.  This  latter  activity  will  pro¬ 
gress  into  the  Operation  phase.  The  applicable  validation  components  are  confidence,  cost,  HCI,  timeliness, 
knowledge  arid  integration.  At  delivery,  these  are  all  nearly  complete,  but  only  after  significant  operational 
use  can  full  validation  be  achieved  due  to  actual  working  practice  evolving  with  exposure  to  the  system,  the 
changing  nature  of  threats  and  requirements  and  the  actual  operational  impact 

VORTEX  obviously  has  no  direct  experience  of  these  activities.  However,  the  progressively  realistic 
development  environment  from  laboratory  to  simulator  to  experimental  aircraft  will  provide  “mini”  -  delivery 
and  operations  pius.-s  within  the  overall  development  phase.  The  mini  delivery  phase  for  the  first  phase  of 
the  Sensor  Advif  .  development  is  described  above.  The  Sensor  Adviser  has  been  widely  demonstrated  since 
then  and  this  has  reinforced  the  current  adequacy  of  the  knowledge  representation  and  timeliness,  as  well  as 
the  need  for  further  development  of  the  scope  and  the  HCI.  This  could  be  viewed  as  a  mini-operation  phase. 

6.  Conclusions 

VORTEX  is  a  project  to  develop  a  methodology  for  building  validated  real-time  decision  aiding  expert 
systems  for  military  operational  use.  It  is  still  under  development  but,  even  if  completed,  it  could  not  hope  to 
provide  a  foolproof  step-by-step  guide  to  building  complex  systems.  Instead  it  seeks  to  provide  a  framework 
for  thinking  about  and  discussing  the  validation  issues  and  provides  checks  and  guidance  for  those  interested 
in  the  development  of  expert  system  modules. 

It  does  this  by  identifying  the  participants  in  the  validation  activity,  the  components  of  validation,  some 
useful  techniques  for  performing  validation  and  a  suitable  life-cycle.  It  attempts  to  fuse  these  together  by  indi¬ 
cating  the  various  participants’  perspectives  on  the  components  at  each  point  within  the  life-cycle  and  the 
appropriate  techniques  to  apply.  Thi  i  is  intended  to  answer  the  questions: 

•  what  should  be  validated? 

•  who  should  do  the  validation? 

•  when  should  validation  take  place? 

•  how  should  validation  be  performed? 

It  is  accepted  that  the  answers  to  the  last  question  are  incomplete,  although  the  framework  presented  here 
may  help  others  to  discover  appropriate  techniques  for  their  particular  applications.  However,  the  methods  that 
we  consider  to  be  generally  applicable  and  of  great  importance  are 

•  use  an  application  specific  knowledge  representation  that  incorporates  vocabulary  and  structure  familiar 

to  application  experts 

•  build  a  feasibility  demonstrator  to  execute  elicited  knowledge  as  early  in  the  development  life-cycle  as 

possible 

•  elicit  and  use  test  cases  which  test  the  boundaries  of  the  expert  system 


•  develop  tools  that  support  validation  and  verification. 

Knowledge  base  inspection  by  the  expert  is  also  expected  to  prove  useful,  although  little  practical  experience 

of  this  has  been  gained. 

This  methodology  has  been  developed  in  conjunction  with  the  development  of  an  ASW  decision  aid  for 

sensor  selection  and  deployment.  Future  work  seeks  to  concentrate  on  validating  the  KB  of  this  application, 

which  will  itself  be  increased  in  complexity  to  develop  the  methodology  for  a  more  realistic  expert  system. 
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SUMMARY 

The  paper  focus  both  theoretical  and  practical  aspects  of  the 
fuzzy  control  systems  according  to  the  following  scope.  First, 
foundations  of  the  fuzzy  controllers,  and  the  different  ways  for 
implementing  them,  are  described.  Second,  we  concentrate  ourselves 
in  the  management  of  the  information,  that  is,  the  way  in  which  the 
inferences  are  made  from  the  expert's  knowledge..  Usually  this  is 
carried  out  by  means  of  the  Generalized  Modus  Ponens  for  which,  the 
so  called  implication  function  is  the  main  tool  to  handle  it.  Hence, 
as  depend  on  the  selected  type  of  implication  one  has  a  different 
version  of  inference,  finally,  the  possible  implications  functions 
and  the  consequences  from  its  use  are  analyzed  and  discussed. 


INTRODUCTION 

The  automatic  control  of  complex  industrial  processes  is  very 
difficult.  The  difficulty  arises  from  non-linearities,  the  time- 
dependent  and/or  ill-defined  behavior  of  systems  and  the  low  quality 
of  available,,  performance  measures.  In  most  real  cases  the  automatic 
control  is  carried  out  through  those  (subsidiary)  variables  that 
accept  measurement  and  control,  whichever  them  may  be.  In  such 
situations  only  partial  goals  may  be  forecasted  by  the  control  model 
and  the  achievement  of  the  global  objectives  are  left  to  the 
human-expert-operators . 

Thus  in  a  lot  of  industrial  processes  the  human  operators  work 
better  than  the  automatic  control  systems.  Their  high  performance 
lies  on  the  experience.  Like  in  other  fields,  the  only  solution  to 
design  and  construct  more  performing  automatic  control  systems,  it 
seems  to  be  the  use  of  knowledge  engineering  to  elucidate  and 
formalize  the  control  strategies,  which  the  human-expert-operators 
often  know  and  justify  from  an  empirical  point  of  view. 

Even  for  well  defined  systems,  usually  the  experts  explain  their 
control  strategies  in  a  linguistic  way,  giving  a  set  of  heuristic 
decision  rules  on  the  time.  Obviously  is  a  very  difficult  (even 
impossible)  task  to  translate  such  linguistic  laws  into  a  set  of 
numerical  ones  without  loss  of  important  pieces  of  knowledge. 
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Obviously  the  problem  increases  when  the  nature  or  the  behavior  of 
the  system  is  ill-defined  or  ill-known.  Thus,  studying  direct 
implementations  of  linguistic  control  rules  is  very  interesting  from 
both  a  theoretical  and  practical  point  of  view. 

The  Theory  of  Fuzzy  Sets  (introduced  by  L.A.  Zadeh  (29]),  and 
more  particularly  the  developments  on  Fuzzy  Reasoning,  provides 
tools  to  represent  and  handle  linguistic  and/or  vague  statement  (for 
instance  rules)  in  an  automatic  manageable  formulation.  The  use  of 
that  tools  in  the  analysis  of  systems,  starts  with  the  Zadeh' s  works 
about  fuzzy  algorithms  and  linguistic  analysis  [30]  and  [32]. 

When  expert's  heuristic  control  rules  are  translated  to  a  fuzzy 
formulation,  that  set  constitutes  a  fuzzy  algorithm  (linguistic 
rather  than  numeric  [32])  which,  in  this  case,  may  be  defined  as 
(logical)  fuzzy  controller.  Fuzzy  control  research  was  started  from 
Mamdani's  pioneer  work  [8].  Since  then,  this  new  approach  foe 
analyzing  and  controlling  complex  systems  has  received  a  growing 
attention. 

Two  different  (but  complementary)  trends  are  traditionally 
distinguished  in  Fuzzy  Control, 

1. -  Analysis:  To  describe  and  model  the  (fuzzy)  behavior  of  a  system 

under  specific  (fuzzy)  constraints  or  (fuzzy)  control 
conditions . 

2. -'  Design:  To  obtain  a  fuzzy  algorithm  being  able  to  control  the 

(fuzzy)  behavior  of  an  specified  (fuzzy)  system. 

Historically,  the  first  works  about  analysis  deal  with  construct¬ 
ing  suitable  theoretic  models,  as  well  as  defining  appropriated 
performance  measures.  As  the  researchers  on  fuzzy  control  arrived  at 
from  classical  Control  Theory,  the  earliest  approaches  extend  to  the 
new  context  the  concepts  of  state,  controllability  and  stability, 
giving  fittest  versions  to  the  Optimal  Control  Problem. 

Further  works  attempt  to  generalize  some  aspects  of  the  classical 
control  models.  Obviously  controlled  fuzzy  systems  may  be  outlined, 
like  the  non-fuzzy  one,  according  to  the  diagram  in  Fig.  1, 


Controller 


inputs 


Process 


^outputs 


(Fig.  1) 


Usually  the  system's  outputs  produce  non-fuzzy  inputs  for  the 
controller,  these  signals  being  obtained  by  means  of  appropriated 
sensors.  In  its  turn,  the  controller  responses  need  to  be  changed 
into  non  fuzzy  signals,  and  then  they  are  inputs  to  the  system. 

Two  areas  of  research  may  be  distinguished  according  to  the  place 
of  the  interfaces  between  the  process  and  its  controller.  When  the 
interfaces  are  inside  the  controller,  then  this  is  just  a  kind  of 
discrete  look-up  table.  The  cases  in  which  the  interfaces  are  into 
the  process  correspond  to  truly  fuzzy  controllers,  and  present  more 
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interesting  properties  than  the  former  ones  (see  [20]). 

Although  works  about  analysis  we(re  largely  motivated  by  practical 
problems,  they  are  quite  theoretic  themselves.  The  design  trend 
however  has  a  more  practical  character.  It  began  with  Mamdani's  work 
[8],  and  its  main  aim  is  to  develop  a  methodology  to  specify  and 
model  fuzzy  control  rules,  in  order  to  construct  and  implement 
effective  fuzzy  controllers. 

The  remain  of  the  paper  is  mainly  devoted  to  different  aspects  of 
the  design,  paying  an  special  attention  to  the  models  of  Approximate 
Reasoning  which  underlies  any  fuzzy  algorithm. 


DESIGN  AND  IMPLEMENTATION  OF  A  FUZZY  CONTROLLER 

Five  components  may  be  distinguished  in  any  fuzzy  controller 
(Fig. 2): 

1)  A  Rule-Base  containing  the  control  rules  or  statements. 

2)  A  Database  which  contains  the  operational  definitions  of  the 
fuzzy  sets  appearing  in  the  fuzzy  rules. 

3)  A  Condition  interface  receiving  non-fuzzy  signals  from  the 
process  and  transforming  them  into  a  fuzzy  set  form. 

4)  An  Action  interface  which  translates  the  controller's  (fuzzy) 
response  into  a  non  fuzzy  action,  and 

5)  A  Computation  unit  inside  which  the  fuzzy  action  is  computed  from 
the  fuzzy  input  signals  provided  by  the  condition  interface. 

It  is  clear  the  most  difficult  tasks  are  to  construct  the  Rule 
Base  the  Computation  Unit  and  the  Database. 


Fig. 2  i 

There  is  no  systematic  knowledge  engineering  procedure  to  obtain 
the  control  rules  to  construct  a  "complete  Rule  Base",  that  is,  one 
being  able  to  give  the  control  action  against  any  signal  from  the 
current  process.  Most  part  of  applications  uses  a  trial-and-error 
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approach  ([2],  [4]).  Some  attempt  to  develop  automatic  procedures 
for  this  task  may  be  found  in  the  literature  ([5],  [10]).  Anyway, 
three  modes  of  derivation  of  fuzzy  control  rules  may  be  distin¬ 
guished  ([16],  [14]): 

1.  Based  on  operator's  experience  and/or  the  control  engineer's 
knowledge . 

2.  Based  on  the  fuzzy  model  of  the  process  controlled,  and 

3.  Based  on  operator's  control  actions. 

The  first  method  is  the  most  used  because  it  is  well-fitted  with 
knowledge  engineering  approaches.  Any  case  the  rules  usually  take 
the  form  of  "if-then"  statements,  with  fuzzy  data  in  the  antecedents 
and  consequences  of  every  implication. 

Another  part  of  the  fuzzy  controller  which  has  received  an 
special  attention,  is  the  computation  unit.  Its  functioning  may  be 
outlined  in  the  following  feedback  loop. 


Input 

System 

Output 

Output  signal 

1 

r 

Input  signal 

— 

If  vague  output  then  vague  input 

i— 

But,  usually  the  controller  contains  several  rules  and  thus 
(denoting  the  relation  if.,  then.,  by  an  arrow)  the  feedback  loop 
may  be  represented  as  in  Fig.  3. 


Fig..  3 


As  the  inputs  and  the  outputs  signals  in  the  computation  unit  are 
fuzzy  or  vague  statements  or  properties,  its  functioning  may  be  just 
seen  as  an  instance  of  Approximate  Reasoning.  Thus,  the  computation 
unit  may  be  seen  as  a  device  which  makes  inferences  to  obtain  the 
outputs  signals  (input  for  the  system)  from  input  signals  (output  of 
the  system)  and  rules.  In  the  literature  two  models  of  computation 
unit  may  be  found  associated  with  the  two  existing  approaches  of 
fuzzy  reasoning: 

a)  Based  on  compositional  rules  of  inference  (t eneralized  "modus 
ponens")  [32,  33,  21,  22]. 

b)  Based  on  fuzzy  logic,  for  instance  fuzzified  Lukasiewicz  loaic 
[22,  23,  1]. 
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Most  part  of  existing  fuzzy  controllers  use  generalized  "modus 
ponens"  (implemented  by  means  of  compositional  rules  of  inference) 
for  the  sake  of  simplicity  in  computation  [15). 

Another  crucial  task  in  the  design  of  a  fuzzy  controller  is  to 
establish  the  Data-  base,  which  involves  determining  the  universe  of 
discourse  of  fuzzy  variables  in  the  input  and  the  output,  as  well  as 
their  semantics,  that  is  the  term  set  of  linguistic  assessments,  and 
the  membership  functions  of  the  fuzzy  sets  representing  that  labels. 
With  respect  to  this  task  two  approaches  may  be  found  in  the 
literature . 

Since  the  Mamdani's  pioneer  works  [8,  9],  the  fuzzy  controllers 
based  on  generalized  "modus  ponens"  usually  assess  the  fuzzy 
variables  on  a  term  set  with  seven  labels: 

NB:  Negative  Big,  NM:  Negative  Medium,  NS:  Negative  Small 
ZO:  About  Zero,  PS:  Positive  Small,  PM:  Positive  Medium 
and  PB:  Positive  Big 

whose  semantics  is  different  according  to  the  characteristics  of  the 
universe  (discrete  or  continuous  [15]). 

On  their  turn,  the  authors  using  fuzzy  logic  assess  variables  on 
a  three  labels  term  set: 

N:  Negative,  ZO:  Zero,  and  P:  Positive 
with  monotonic  membership  functions  (see  [15]), 

In  following  sections  we  will  specially  turn  our  attention  to  the 
fuzzy  controllers  which  use  generalized  "modus  ponens". 

The  final  step  in  the  construction  of  a  fuzzy  controller  is  the 
effective  implementation  of  its  design.  Like  the  classical  control 
model  implementations,  there  exist  two  different  ways  for  implement¬ 
ing  a  fuzzy  controller:  1)  by  means  of  software  systems,  and  2) 

through  hardware  devices . 

In  the  first  case,  the  central  task  is  to  write  computer  programs 
(in  a  suitable  programming  language)  being  able  to  make  the 
aforementioned  inferences.  Some  kind  of  suitable  interfaces  between 
computer  and  system  are  obviously  needed.  Most  part  of  works  in 
fuzzy  control  are  in  this  way  (see  references  about  applications). 

In  the  second  approach  the  central  task  is  to  construct  a 

hardware  device  with  the  specific  purpose  of  making  the  control 
inferences.  Suitable  interfaces  with  the  system  are  needed  again, 
but  their  nature  is  quite  different  to  the  ones  in  the  software 
approach . 

One  of  the  most  active  researchers  in  the  field  of  hardware 

implementations  of  fuzzy  controllers  is  T.  Yamakawa,  [24,  25,  26] 

which  has  constructed  the  so  called  "Rules  Chip" ,  a  chip  that 

implements  control  inferences  using  Mamdani's  compositional  rule  of 
inference  (see  next  section)  to  model  the  generalized  "modus 
ponens".  We  must  point  out  Yamakawa 's  research  goals  are  beyond 
fuzzy  control,  as  this  author  seeks  for  designing  and  constructing  a 
"Fuzzy  Computer",  that  is,  an  electronic  device  having  the  inference 
as  intrinsic  operation,  like  digital  computer  are  designed  to  make 
arithmetic  [25,  27,  28]. 
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Hardware  implementations  of  fuzzy  controllers,  and  more  generally 
of  fuzzy  inferences  are  receiving  a  growing  attention  [see  for 
instance  17,  18,  19,  6]. 

GENERALIZED  "MODUS  FONENS" 

Systems  using  internal  representations  of  knowledge  need  to  have 
the  ability  of  making  inferences,  that  is,  they  must  be  able  to 
obtain  new  facts  from  already  known  ones. 

The  "modus  ponens"  (the  basic  deduction  rule  from  Predicate 
Calculus)  is  the  best  known  inference  method,  and  it  has  been 
extensively  used  in  Artificial  Intelligence  developments.  It  may  be 
established  as  follow. 

Assumed  that  the  implication  if  P  then  Q  is  true,  and  given  that 
P  happens  (that  is,  the  fact  or  proposition  P  is  true),  then  one 
must  conclude  the  fact  (proposition)  Q  is  true.  Usually  this  rule  of 
inference  is  denoted  as 


P - »  Q 

P _ 

Q 

In  most  cases  P  and  Q  capture  knowledge  about  variables.  The 
simplest  instance  consider  P  and  Q  are  statements  about  the 
variables  x  and  y  respectively.  That  is,  P  and  Q  stand  for  "x  is  A" 
and  "y  is  B"  respectively,  where  A  and  B  represent  properties  of  x 
and  y,  i.e.  subsets  of  the  corresponding  universes  of  discourse, 
whith  such  formulation,  the  "modus  ponens  is  written  as 

x  is  A - >  y  is  B 

x  is  A _ 

y  is  B 

The  deduction  rule  in  that  cases,  where  P  and/or  Q  involve 
several  variables,  is  straightforward  obtained. 

Usually  expert's  rules  or  statement  about  facts  are  vague, 

imprecise  or  fuzzy,  rather  than  exact  or  precise.  Inference  methods 

for  that  cases  are  obviously  needed  if  one  is  looking  for  modeling 
human  reasoning  to  implement  it  in  an  automatic  system.  In  the 

setting  of  Approximate  Reasoning,  Zadeh  [32],  introduced  an 
extension  of  the  classical  "modus  ponens",  that  is  known  as 
"generalized  modus  ponens",  to  deal  with  fuzzy  propositions  or  fuzzy 
statements  about  variables,  i.e.  fuzzy  or  linguistic  variables. 

Let  us  consider  a  rule  if  x  is  A  then  y  is  B,  where  A  and  B  are 
fuzzy  properties  (constraints)  about  the  variables  x  and  y  respect¬ 
ively,  i.e.  fuzzy  subsets  of  the  corresponding  universes  of 
discourse.  Moreover,  let  us  suppose  an  observation  about  x  that 

allows  to  state  x  is  A' ,  where  A'  is  a  fuzzy  statement  about  x  being 
related,  but  in  general  different,  to  A.  By  generalizing  the 
classical  scheme,  the  conclusion  must  be  in  the  form  y  is  B' 

x  is  A  - >  y  is  B 

x  is  A' 


y  is  B' 
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where  B'  is  to  be  a  fuzzy  set  on  the  universe  of  y,  related,  but  in 
general  different,  to  B.  There  the  key  problem  is  to  effectively 
determine  B' . 

The  most  usual  approach  to  obtain  B'  is  the  so  called  "Composi¬ 
tional  Rule  of  Inference"  developed  by  Zadeh  [30,  31,  32].  The  very 
idea  is  the  rule  "x  is  A  — >  y  is  B"  induces  a  fuzzy  relation  R 
between  x  and  y,  that  is,  a  fuzzy  subset  in  the  cartesian  product  of 
the  corresponding  universes  of  discourse  whose  membership  function 
depend  upon  the  membership  functions  of  A  and  B: 

mR(x,y)  =  F[mA(x),mB(y)] 

and  then  B'  is  the  transformation  of  B  through  R,  i.e. 

B'  =  R  ®  B 

where  ®  denotes  some  composition  operation. 

Let  us  observe  F(a,b)  is  just  an  implication  function  in  the 
classical  sense  of  the  logic. 

According  to  Zadeh' s  Extension  Principle  we  may  write 

mB,(y)  =  Max  (mA, (x)  *  mR(x,y)) 

*  being  an  associative  and  monotonous  application  from  [0,l]x[0,l] 
to  [0,1],  i.e.  a  t-norm,  and  in  this  way  the  problem  of  determining 
B'  has  been  changed  into  establishing  F  and  *. 

Several  options  are  possible  and  each  conveys  to  a  different 
instance  of  "generalized  modus  ponens"  i.e.  to  a  different  approxi¬ 
mate  reasoning  model,  which  may  be  use  to  model  human  ways  of 
reasoning.  Next  section  deals  with  a  comparative  study  of  several 
choices  of  F  and 

Roughly  speaking  we  can  say  the  control  unit  of  a  fuzzy  control¬ 
ler  may  be  designed  just  as  a  device  that  performs  some  model  of 
"generalized  modus  ponens".  To  illustrate  its  operation,  let  us 
consider  a  system  for  which  the  control  rules  are 

If  x1  is  A1;l  and  x2  is  A^2  then  y  is  B1 

If  x1  is  A21  and  x2  is  A22  then  y  is  B2 

A^  and  B^ ,  being  labels  belonging  to  the  term  sets  where  the 

variables  x ^  and  y  are  assessed,  i,j  =  1,2  (usually  as  we  have 

described  in  last  section).  Moreover,  let  us  consider  that  Mamdani's 
version  of  the  compositional  rule  of  inference  is  used  [8,  9],  that 
is 

F(a,b)  =  min  {  a,b  > 

*  =  Min 

If  the  (non-fuzzy)  signals  from  the  system  (obtained  by  means  of 
a  suitable  sensor)  are  x£,  x2,  belonging  to  the  universe  of 

discourse  of  x^  and  x2  respectively,  then  the  compatibility  of  these 

observation  with  the  premises  of  each  control  rule  is  given  by 
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wx  =  ^  Ai2 ^ x2 ^ 

w2  =  ^21^xl^  A  ^22^X2^ 

and  thus  the  conclusion  B'  must  be  characterized  by 
B'(y)  =  Max  {v^A  B^y),  w2A  B2(y)} 

Let  us  observe  the  output  from  the  computation  unit  is  a  fuzzy 
subset  in  the  universe  of  discourse  of  y.  For  control,  a  crisp 
response  being  applicable  to  the  systems  is  needed,  and  then  B' 
ought  be  defuzzified  before  its  going  out  of  the  fuzzy  controller. 
Synthesizing  B'  in  its  gravity  center  is  quite  usual. 


USING  DIFFERENT  IMPLICATION  FUNCTIONS  IN  THE  GENERALIZED  "MODUS 
PONENS" 

In  {32]  Zadeh  proposes  to  use  the  Min  operator  as  *,  and  F  the 
Lukasiewicz's  implication  function,  i.e.  F(a,b)  =  min  {  1,1-a+b} 

In  [11,  12,  13]  the  behavior  of  the  “generalized  modus  ponens" 
under  other  different  implication  functions  (keeping  *  =  Min)  are 
dealt  with.  These  studies  have  a  theoretical  character  rather  than  a 
practical  one. 

At  present,  our  own  research  is  mainly  concerned  with  the 
possibility  of  designing  hardware  devices,  able  to  make  inferences 
by  means  of  "generalized  modus  ponens".  Thus  we  are  looking  for 
implication  functions  and/or  t-norms  with  two  key  features: 

a)  they  must  produce  an  inference  method  representing  an  average 

human  reasoning, 

b)  they  must  be  simple  enough  to  be  implemented  through  electronic 

circuits. 

Moreover,  for  a  formal  coherence  principle,  it  seems  reasonable 
to  want  obtain  output  with  the  same  characteristics  than  the  input, 
in  the  sense  that  if  the  input  is,  for  instance,  a  trapezoidal  fuzzy 
number  (tfn),  the  corresponding  output  must  also  be  another  tfn. 

With  these  goals  in  mind,  we  are  carrying  out  studies  rather 
different  to  the  aforementioned  ones,  paying  an  special  attention  to 
the  behavior  of  "modus  ponens”  in  relation  with  points  a)  and  b) 
below..  In  the  following,  we  summarize  a  simulation  experiment  which 
attempt  to  discover  the  possibility  of  B'  preserving  with  respect  to 
B,  the  relative  position  of  A'  with  respect  to  A,  under  different 
implications  functions. 

We  have  assumed,  on  a  hand,  X  and  Y,  the  universes  of  discourse 
of  the  variables  x  and  y  respectively,  are  closed  and  bounded 
intervals  of  the  real  line.  Thus,  both  X  and  Y  may  be  taken  equal  to 
(a ,b]  or  more  particularly  X  =  Y  =  [0,1] .  On  the  other  hand  every 
memoership  function  (ms [0,1]  — »  [0,1])  is  sampled  on  30  equally 
spaced  points,  to  obtain  an  internal  computer  representation.  Each 
sampled  grade  of  membership  is,  of  course,  ranging  from  0  to  1 . 
Thus,  any  fuzzy  set  is  represented  by  a  vector  of  30  elements,  that 
is  a  point  in  ( 0 , 1 ] 30 ,  and  we  can  write: 
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A  a 

(ar  .  , 

,  .  .a3(J) 

with 

ai 

=  vxi) 

A'  a 

(ai  •  • 

■  • *a3o) 

with 

ai 

=  ">A,(x.) 

B  a 

(br* 

’••b30> 

with 

bj 

=  mB ( y  j ) 

Obviously  the  relation  R  appears  as  a  30x30-matrix  defined  by 

r..  =  F(a.,b.)  and  thus 
i]  1  i'  ] ' 

B'  a  (b^ . b^)  with  bj  =  max  (a|  A  r^) 

In  our  experiment,  A,  A'  and  B'  are  supposed  tfn,  because  they 

are  simple  to  handle,  and  good  enough  to  represent  any  kind  of 

vagueness  or  uncertainty  of  data.  With  respect  to  the  relative 

position  of  A  and  A',  three  cases  have  been  considered:  1)  A'  is 
included  in  A,  2)  A'  is  included  in  A,  but  A' and  A  have  a  non  empty 
intersection  and  3)  A  is  included  in  A' 

Fifteen  different  implication  functions  have  been  considered  to 
define  R.  They  are  the  most  usual  ones  [13],  and  are  listed  in  the 
first  column  of  the  table  1. 

Table  1  summarizes  the  results  of  our  experiment.  Each  row 

presents  the  assessment  of  the  behavior  of  each  implication 

function,  against  the  aforementioned  three  cases  of  relative 

position  for  A  and  A' ,  in  such  a  way  that  any  pigeonhole  contains 
three  labels  LI,  L2,  L3,  with  the  following  characteristics: 

A)  LI  measures  the  goodness  of  F  in  preserving,  on  B'  and  B,  the 
relative  position  of  A'  with  respect  to  A.  The  possible  linguis¬ 
tic  values  we  have  used  are  vb  =  very  bad,  b  =  bad,  m  =  medium, 
g  =  good  and  vg  =  very  good 

B)  L2  assess  the  increment  of  fuzziness  of  B'  with  respect  to  B, 
measured  through  the  value  I  =  min^bj*.  j  =  1,2,. ..,30  }.  The  used 

linguistic  values  are  E  =  non  increment  (I  =  0),  F  =  fair 

increment  (I  <  0.5),  UF  =  unfair  increment  (I  >  =  0.5),  T  =  total 

fuzziness  (1=1) 

C)  L3  is  a  boolean  parameter  that  indicates  whether  B'  is  (Y)  or  not 
(N)  a  tfn.. 
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ABSTRACT 

With  the  advent  of  the  USAF’s  Advanced  Tactical  Fighter  and  NASA’s  National  Aerospace  Plane,  demands 
for  concise  test  data  reduction  and  interpretation  will  increase  beyond  the  capabilities  of  current  methodologies 
As  mission  complexity  increases  it  becomes  apparent  that  real  time  data  analysis  for  flight  safety,  mission  control 
and  test  conduct  becomes  a  necessary  tool. 

A  neural  network  is  a  biologically  inspired  mathematical  model,  which  can  be  represented  by  a  directed 
graph,  that  has  the  ability  to  learn  through  training.  Neural  networks  have  many  advantages  over  current 
aviation  computing  systems  including  the  ability  to  learn  and  generalize  from  their  environment.  Neural 
networks  arc  excellent  for  parameter  estimation  and  recognizing  patterns  in  signal  data  This  research  discusses 
a  prototype  system  designed  and  implemented  at  the  University  of  Tennessee  Space  Institute  to  discover  patterns 
in  test  data  from  an  engine  test  cell  in  order  to  determine  if  any  part  of  the  system  is  in  failure.  The  results  of 
this  research  show  that  a  neural  network  can  be  used  for  fault  diagnosis  in  an  engine  test  <.  'll  when  the  problem 
of  fault  monitoring  and  diagnosis  is  seen  as  one  of  pattern  recognition  A  two  layer  senuhnear  feed-forward 
neural  net  is  able  to  separate  simulated  sensor  data  into  normal  and  abnormal  classes  and  the  addition  of  a 
hidden  layer  mokes  the  network  more  resistant  to  noise  and  improves  the  ability  of  the  network  to  classify  the 
type  of  fault  that  is  occuring. 

Neural  networks  offer  test  peisonnel  a  new  tool  for  the  analysis  of  critical  test  data.  A  general  working 
network  model  can  be  applied  to  any  system  where  sensor  datu  is  evaluated  and  fault  diagnosis  is  critical 
Significant  benefit  to  the  aerospace  community  is  gai.  .ed  by  improved  safety  of  test  engineers  by  rapid  detection 
of  faulty  conditions.  This  will  be  of  extreme  importance  as  testing  begins  on  more  advanced  engine  systems 
such  as  the  National  Aerospace  Plane  and  the  Advanced  Tactical  Fighter 


(1.0)  INTRODUCTION 


(1.1)  Problem  Statement 

Simulated  flight  tests  conducted  in  an  engine  test  cell  produce  large  quantities  of  sensor  data  at  rates  that 
overwhelm  flight  test  personnel.  The  task  of  monitoring  signals  from  test  cell  sensors  and  watching  for  tile 
onset  of  possible  fault  situations  requires  significant  experience  and  a  number  of  test  cell  personnel  Because 
of  the  variety  and  quantity  of  test  cell  data,  extensive  knowledge  and  experience  is  required  for  accurate 
differentiation  of  data  types.  A  neural  network  architecture  that  could  accurately  diagnose  the  onset  of  fault 
situations  and  alert  test  cell  engineers  would  result  in  a  significant  savings  in  personnel,  downtime  because  of 
damaged  equipment,  and  improve  test  cell  safety, 

(1.2)  Approach 

The  main  approach  of  this  research  is  to  design  and  analyze  a  network  architecture  to  monitor  sensor  data 
from  an  engine  test  cell  to  detect  data  that  falls  outside  learned  parameters,  thus  signifying  a  fault.  In  this 
research,  diagnostic  problems  in  genera!  are  conceptualized  as  a  mapping  of  an  input  pattern  (representing 
sensor  data)  to  an  output  pattern  which  is  interpretable  as  a  diagnosis. 


*©1990.  This  research  was  supported  by  Sverdrup  Technology  under  contract  number  1102*400489  All 
references  to  the  Aeropropulsion  Systems  Test  Facility  (ASTF)  arc  from  public  domain  documents  (sec  Polce  s 
MS  Thesis).  This  paper  was  cleared  for  public  release  by  USAF/DOCS  and  USAF/PA  on  20  Feb  1990. 


A  neural  network  is  a  biologically  inspired  mathematical  model,  which  can  be  represented  by  a  directed 
gtaph,  that  has  the  ability  to  learn  through  training.  The  action  of  a  neural  net  may  be  viewed  principally  as 
a  mapping  through  which  points  in  the  input  space  are  transformed  into  corresponding  points  in  the  output 
space  on  the  basis  of  designated  attribute  values  (of  which  class  membership  might  be  one)  (Pao,1989).  It  is 
the  goal  of  this  research  to  design  a  neural  network  arrliitccture  that  can  classify  test  data  as  part  of  a  nominal 
engine  test  or  as  data  from  a  system  in  failure.  The  words  neural  network,  network,  net,  and  network  model 
will  be  used  interchangeably  throughout  this  thesis. 


(2.0)  PROBLEM  ANALYSIS 


(2.1)  The  ASTF  Domain 

The  Aeropropulsion  Systems  Test  Facility  (ASTF)  is  designed  for  testing  airbreathing  engine  propulsion 
systems  over  a  wide  range  of  altitudes  and  Mach  numbers.  Simulated  flight  tests  conducted  in  the  facility  will 
determine  the  operational  and  performance  characteristics  of  aeropropulsion  systems;  thus,  the  time  required 
for  flight  tests  is  shortened,  and  the  risks  and  expense  of  flight  tests  are  minimized  (Poke,  1982). 

The  ASTF  was  designed  to  permit  development  and  surveillance  tests  of  large  airbreathing,  jet  propulsion 
engines  throughout  their  operational  envelopes  by  providing  the  capability  for  true  simulation  of  flight  altitude 
and  flight  Mach  number  conditions.  The  facility  is  capable  of  performing  direct-connect  testing  of  turbojet  and 
high-bypass  (8:1)  turbofan  engines  up  to  75,000  pound-rated,  sea  level,  static  thrust  throughout  the  appropriate 
engine  operating  envelopes  (Poke,  1982).  The  facility  is  designed  to  allow  testing  in  both  direct-connect  and 
freejet  modes. 

In  Poke's  (1982)  thesis  he  describes  different  types  of  tests  that  will  be  conducted  in  the  ASTF.  These 
test  types  require  the  capability  of  transient  testing,  e.g.,  engine  power  transients,  flight  profile  transients,  or 
combinations  of  the  two.  The  facility  operating  conditions  and  the  corresponding  engine  operating  conditions 
for  testing  in  the  ASTF  are: 

(1)  Steady  Environment  —  Steady  Engine  Power  Setting. 

(2)  Steady  Environment  —  Transient  Engine  Power  Setting. 

(3)  TVansient  Environment  —  Steady  Engine  Power  Setting. 

(4)  TVansient  Environment  —  TVansient  Engine  Power  Setting  (Mission  Profile  Simulation)  (Poke,  1982). 

Looking  at  the  testing  requirements  that  the  ASTF  must  perform,  it  becomes  obvious  that  systems  within 
the  test  cell  must  be  able  to  alter  both  pressures  and  temperatures,  along  with  a  number  of  valve  openings, 
to  regulate  test  cell  conditions.  As  the  system  increases  in  complexity,  it  becomes  an  overwhelming  task 
to  supervise  all  the  sensors  that  monitor  the  facility.  Instrumentation  and  data  handling  equipment  must 
be  provided  to  measure,  transmit,  acquire,  display,  and  process  those  measurements  which  are  required  to 
determine  the  performance  of  the  test  article.  The  measurement  capability  for  each  test  cell  includes  970 
pressure  measurements,  820  temperature  measurements,  and  152  miscellaneous  measurements,  such  as  forces, 
flows,  speeds,  geometries,  vibrations,  accelerations,  and  strains  (Polce,  1982). 

Three  major  subsystems  make  up  the  ASTF  instrumentation  and  control  system  They  arc  the  Plant 
Instrumentation  and  Control  System  (PICS),  The  Test  Instrumentation  System  (TIS)  and  the  Automatic  Test 
Control  System  (ATCS). 

The  ATCS  performs  a  number  of  control  functions  inside  the  test  cell.  The  ATCS  is  used  to: 

(1)  Control  test  conditions. 

(2)  Control  engine  operating  conditions. 

(3)  Coordinate  control  of  test  and  engine  operating  conditions  with  plant  configuration  changes 

(4)  Detect  abnormal  plant  and  engine  operating  conditions  and  their  response  thereto 

(5)  Communicate  with  facility  operators  (Poke,  1982) 

Because  of  the  importance  of  the  ATCS  and  because  it  is  used  for  detection  of  abnormal  conditions  inside 
the  test  cell,  this  research  concentrates  on  applying  neural  network  technology  to  enhancing  the  ATCS 

The  ATCS  controls  the  parameters  which  establish  a  desired  test  condition  by  positioning  forty-six  of  the 
facility  configuration  and  control  valves.  The  ATCS  is  used  to  control  various  engine  controls  and  the  interface 
to  the  test  cell  engineers  is  effected  at  a  panel  in  the  test  building  control  room.  Pertinent  test  information  is 
communicated  to  test  cell  personnel  through  four  CRT  terminals.  It  is  the  end  goal  of  this  lesemcii  to  establish 
a  better  means  of  monitoring  test  data  from  the  ATCS. 
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(2.2)  Fault  Monitoring  and  Diagnosis 

In  order  to  apply  neural  networks  to  fault  monitoring  and  diagnosis,  the  task  of  fault  detection  must  be 
seen  as  one  of  pattern  recognition.  In  approad.ng  the  ASTF  we  will  say  that  fault  detection  is  observing  data 
that  fall  outside  expected  parameters  previously  regarded  as  “normal"  operation.  Sensor  data  that  fall  outside 
nonnal  boundaries  do  not  necessarily  indicate  that  the  system  is  beginning  to  fail,  but  it  does  indicate  that  an 
abnormal  occurrence  is  taking  place  inside  the  system  that  is  being  studied  and  warrants  further  attention.  As 
stated  previously,  in  this  research  diagnostic  problems  are  viewed  as  a  mapping  of  an  input  pattern  representing 
sensor  data  to  an  output  pattern  which  is  interpietablc  as  a  diagnosis. 

Analysis  of  a  complex  system  should  use  an  opportunistic  strategy  —  there  is  no  computationally  feasible 
“legal  move  generator”  that  defines  the  space  of  solutions  in  which  pruning  and  steering  take  place  as  in  classical 
AI  systems  (Nii  and  Fcigenbaum,  1978).  Bits  and  pieces  of  information  must  be  used  as  the  opportunity  arises 
to  (relatively)  slowly  build  a  coherent  picture  of  the  world  (system).  This  is  one  way  in  which  neural  networks 
are  better  than  current  AI  methods  at  analyzing  signal  data  from  complex  systems. 


(3.0)  THEORETICAL  DEVELOPMENT 


(3.1)  Neural  Networks 

As  stated  previously,  we  will  define  a  neural  network  as  a  biologically  inspired  mathematical  model,  which 
can  be  represented  by  a  directed  graph,  that  has  the  ability  to  learn  through  training  A  neural  network  model 
typically  has  eight  major  aspects.  They  arc- 

(1)  A  set  of  processing  units 

(2)  A  state  of  activation 

(3)  An  output  function  for  each  unit 

(4)  A  pattern  of  connectivity  among  the  units 

(5)  A  propagation  rule  for  propagating  patterns  of  activities  through  the  network  of  connectivities. 

(6)  An  activation  rule  for  combining  inputs  to  produce  a  new  activation  level  for  a  unit. 

(7)  A  learning  rule  whereby  ,  terns  of  connectivity  are  modified  by  experience. 

(8)  An  environment  within  which  the  system  must  operate  (Rumelhart  and  McClelland,  1988) 

These  will  each  be  described  in  detail  below 

Each  processing  unit  in  a  neural  network  model  represents  some  featui e-like  entity  such  as  an  engine  sensor 
It  is  the  pattern  as  a  whole  coming  from  the  system  to  be  studied  that  is  the  meaningful  level  of  analysis.  The 
purpose  of  a  unit  is  to  receive  input  from  its  neighbors  and  as  a  function  of  the  inputs  i(  receives  to  output  a 
value  to  its  neighbors.  There  are  three  types  of  processing  units  in  a  neural  network  model-  the  input  units 
which  receive  data  from  sources  external  to  the  model,  one  or  more  layers  of  hidden  units  which  are  internal  to 
the  model  only,  and  a  series  of  one  or  more  output  units  which  sends  signals  out  of  the  model  to  effect  motoric 
systems  or  influence  other  externa!  systems.  Figure  1  shows  a  three  layer  feed-forward  neural  network. 


Figure  1  A  Feed-Forward  Neural  Network  Architecture 


A  state  of  activation  is  the  representation  of  the  state  of  the  system  at  some  time  t  representing  the  pattern 
of  activity  over  the  set  of  processing  units  It  is  the  pattern  of  activation  over  the  set  of  units  that  captures 
what  the  system  is  representing  at  any  time 

Associated  with  each  unit  y,  there  is  an  output  function  y  =  wx  which  maps  the  current  state  of  activation 
to  some  output  y,;  y,  =  £ui,,z;.  There  is  often  some  threshold  function  so  that  a  unit  has  no  effect  on  its 

i 

neighboring  unit  unless  the  output  function  exceeds  some  certain  value  (Rumelhart  and  McClelland,  1988). 
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The  pattern  of  connectivity  constitutes  what  the  system  knows  and  determines  how  it  will  respond  to  some 
arbitrary  input.  Neural  networks  aie  often  called  connectionist  networks  because  the  computations  in  the  net 
are  largely  performed  by  the  connection  strengths.  The  signal  coming  into  a  processing  unit  is  the  inputs  from 
all  incoming  units  multiplied  by  a  weight  and  summed  to  get  the  overall  input  to  that  unit,  as  in  Figure  2 

The  total  pattern  of  connectivity  can  be  represented  by  specifying  the  weights  for  each  of  the  connections 
in  the  system.  Positive  and  negative  weights  can  be  seen  as  excitory  or  inhibitory  input  respectively.  The 
absolute  value  of  the  weight  gives  the  strength  of  the  connection.  How  to  determine  appropriate  weights  is 
dependent  on  the  learning  algorithm 

Propagation  in  the  net  is  the  sum  of  the  net  excitory  inputs  (the  weighted  sum  of  the  excitory  inputs  to 
the  unit)  and  the  net  inhibitory  inputs  (the  weighted  sum  of  the  inhibitory  inputs  to  the  unit).  (Rumelhart 
and  McClelland,  1988).  The  activity  of  a  unit  is  regarded  as  the  degree  of  confidence  that  a  preferred  feature 
is  present.  To  reach  a  new  state  of  activation  a  function  /  (activation  rule)  will  be  used  to  take  x  and  produce 
some  new  state  of  activation.  /  should  be  differentiable  in  order  to  use  certain  learning  algorithms 
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Figure  2.  A  Single  Neural  Network  Processing  Element 

What  makes  a  network  useful  in  the  analysis  of  complex  systems  is  the  ability  to  modify  its  connectivity 
through  experience.  There  are  three  types  of  modifications  to  a  network-  the  development  of  new  connections, 
the  loss  of  existing  connections  and  modifications  of  the  strengths  of  connections  that  already  exist  in  the  net 
The  derivation  of  these  modification  (learning)  rules  will  be  covered  under  the  section  on  learning  algoritluns 

The  last  aspect  of  the  model  is  the  environment  in  which  it  will  operate.  Formally,  we  will  define  this  as  a 
time  varying  stochastic  function  over  the  space  of  the  input  patterns.  More  simply;  at  any  point  in  time,  there  is 
some  probability  that  any  of  the  possible  set  of  input  patterns  is  impinging  on  the  input  units  The  environment 
for  this  research  is  the  input  vectors  composed  of  sensor  data  coming  from  the  ATCS  in  the  Aeropropulsion 
Systems  Test  Facility. 

A  description  of  how  a  neural  network  processes  information  follows-  An  input  signal  (stimulus)  is  received 
by  the  nodes  in  the  input  layer  and  produces  some  activation  level  in  a  given  input  unit,  which  conveys  a  signal 
of  proportional  strength  based  on  the  weighting  matrix  to  its  [synaptic)  connections,  or  nodes  in  the  next  layer 
The  global  effect  is  that  a  pattern  of  activation  across  the  set  of  input  units  produces  a  distinct  pattern  of 
activations  across  the  set  of  hidden  units  (Churchland  and  Churchland,  1990).  The  process  is  repeated  again 
as  many  times  as  there  are  hidden  layers.  As  before,  an  activation  pattern  across  the  hidden  units  produces 
a  distinct  activation  pattern  across  the  next  layer  of  nodes  In  this  way  the  network  becomes  a  device  for 
transforming  some  input  vector  representing  sensor  data  into  a  uniquely  corresponding  output  vector.  It  is  a 
device  for  computing  a  specific  function.  Exactly  what  function  it  computes  is  fixed  by  the  global  configuration 
of  its  synaptic  weights  (Churchland  and  Churchland,  1990). 

There  are  a  number  of  procedures  for  training  a  network  and  adjusting  the  weights  Many  learning 
algorithms  have  been  derived  over  the  last  two  decades.  In  choosing  a  learning  algorithm  for  a  system  to 
monitor  the  ASTF  we  must  realize  the  function  that  we  want  to  specify  is  initially  unknown  to  us  because  of 
the  myriad  possible  fault  situations  We  can  impose  an  unspecifyable  function  on  the  network  as  long  as  we 
can  supply  a  set  of  examples  of  the  desired  input/output  pairs.  The  network  is  then  trained  until  it  performs 
the  input  to  output  transformation  desired. 


(3.2)  Learning  Algorithms 


When  feu^ble,  learning  is  a  very  attractive  alternative  to  explicit  programming.  This  is  particularly  true 
when  problems  do  not  lend  themselves  to  systematic  programming,  such  as  pattern  recognition.  AH  neural 
network  learning  aigoritlims  have  their  basis  in  the  neural  learning  rule  disco /ered  by  Hebb  in  1949  His  rule 
for  “Hebbian  learning”  in  a  network  is  as  follows: 

When  an  axion  of  cell  A  is  near  enough  to  excite  a  cell  B 
and  repeatedly  or  persistently  takes  part  in  firing  it,  some 
growth  process  or  metabolic  change  takes  place  in  one  or 
both  cells  such  that  A’s  efficiency,  as  one  of  the  cells  firing 
B,  is  increased 

Neural  Networks  can  be  classified  on  the  basis  of  the  patterns  stored  and  their  method  of  recall  Two 
labels  for  networks  used  as  associative  memories  are  autoassociative  networks  and  heteroassociative  networks 
Autoassociative  networks  associate  a  data  item  with  itself.  Heteroassociative  networks  associate  two  different 
data  items  —  one  is  used  to  recall  the  other  Hetcioassociative  type  memories  must  be  trained  with  a  data  pan 
representing  the  input  and  the  desired  output  pattern  associated  with  it.  To  store  the  pattern  in  a  training  set 
each  input  is  presented  to  the  system  along  with  the  desired  output  and  some  learning  rule  is  used  to  adjust  the 
weights  usually  based  on  a  difference  equation  and  error  between  actual  and  desired  outputs  To  recall  stored 
information  an  unfamiliar  pattern  from  a  different  set  of  patterns  is  presented  to  the  network  and  the  system 
outputs  the  second  member  of  the  training  pair,  the  associated  pattern  of  output. 

A  heteroassociative  network  will  be  used  for  fault  monitoring  of  the  ASTF  Interpretation  of  the  network’s 
outputs  and  weights  should  give  insight  into  the  meaning  of  signals  in  a  complex  sysiem  If  a  network  is  able 
to  respond  to  a  set  of  “interesting”  patterns  then  this  will  form  the  basis  for  a  feature  detector  which  we  will 
use  for  detecting  faults  in  our  complex  system. 

The  learning  mie  used  for  this  3ystem  to  discover  normal  and  abnormal  patterns  of  data  from  the  ASTF  is 
backpropagation  of  enor  which  was  discovered  simultaneously  by  a  number  of  different  research  groups  in  the 
early  1980s.  Backpropagation  is  a  hetcioassociative,  function  estimating,  neural  network  that  stores  arbitrary 
analog  spatial  pattern  pairs,  using  multi-layer  crroi -correction  encoding  between  a  target  output  vector  and 
actual  output  values  from  the  net 

The  learning  procedure  consists  of  the  net  being  assigned  a  random  set  of  we;.,!  is  and  icccivmg  a  training 
set  pair  as  input  and  evaluating  the  output  in  a  feedforward  manner  The  initial  difference  error  between  target 
and  actual  outputs  will  be  quite  large.  Using  the  backpropagation  procedure,  the  algorithm  calculates  a  change 
m  weight  Atv4;  for  all  tv,,  in  the  net  for  that  particular  pattern  This  procedure  is  repeated  for  all  patterns  in 
the  training  set  to  yield  a  resulting  A wtJ. 


Aui,;  =  r)6,j{q)yj(q) 

where  6>s(q) z”r*r(q)  •  *•«(*)  jl  -  z™\q)\ 

zdt*irc<f.  ls  desired  output  for  node  c,  and  is  the  actual  output  value  of  node 

Corrections  are  made  to  the  weights  and  the  outputs  again  are  evaluated  in  a  feedforward  manner  based 
on  the  number  of  iterations  specified  by  the  system.  Discrepancies  between  actual  and  target  output  values 
again  result  in  evaluation  of  weight  changes  (Pao,  1989).  If  the  training  is  successful,  the  system  error  will 
decrease  with  the  number  of  iterations  the  system  performs  and  the  procedure  will  converge  to  a  stable  set  of 
connection  weights.  For  a  complete  derivation  of  the  backpiopagation  learning  rule  sec  Appendix  B:  Derivation 
of  the  Backpropagation  Learning  Rule. 

Backpropagation  is  not  guaranteed  to  find  the  global  error  minimum,  only  the  local  error  minimum  which 
should  be  good  in  enough  in  determining  a  fault  situation  in  the  ASTF  facility.  Backpropagation  was  chosen 
for  this  research  because  of  its  ability  to  store  many  patterns,  its  ability  to  generalize  and  its  ability  to  perform 
arbitrary  nonlinear  mappings  (Simpson,  1990). 


(3.3)  The  Network’s  Ability  to  Generalize  from  Example 

When  considering  neural  networks  for  fault  monitoring  and  diagnosis  it  is  important  to  consider  how  well 
;hc  network  can  generalize  from  the  set  of  training  tramples  It  is  impossible  (and  impractical)  to  present  the 
network  with  all  possible  examples  of  a  fault  We  are  expecting  the  network  to  learn  from  a  set  of  examples 
and  be  able  to  generalize  for  i  ll  possible  faults. 
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In  diagnosing  faults  in  the  ASTF  we  have  an  environment,  which  is  a  set  of  sensor  values,  and  call  this  set 
X.  In  this  environment  we  have  a  concept  defined  as  a  function  g  •  x  -*  {0, 1}  which  is  the  presence  or  absence 
of  a  fault.  The  goal  of  learning  is  to  produce  a  hypothesis  h  :  x  -♦  {0,1}  that  approximates  the  concept  g 
(Abu-Mostafa,  1989).  In  our  case  h  is  a  pattern  recognition  system  that  recognizes  the  presence  of  faulty  data. 
To  do  this  we  have  a  number  of  examples  of  the  concept  (*i,<7(xi))...(xn,0(xn)),  case  a  ^  °f  input 

vectors  from  the  ASTF  simulator  and  the  desired  outputs.  A  learning  algorithm  is  one  that  takes  the  examples 
and  produces  the  hypothesis.  Performance  of  the  system  can  be  measured  by  the  number  of  examples  needed 
to  produce  *»  good  hypothesis. 

When  backpropagation  is  used  in  a  feed  forward  manner,  we  are  guessing  that  there  is  a  good  hypothesis 
to  be  gained  by  setting  the  weights  and  thresholds  of  the  network.  The  set  of  hypothesis  H  would  then  be  the 
set  of  all  functions  h  that  are  obtained  by  setting  the  weights  and  thresholds  of  the  net.  The  learning  algorithm 
picks  a  hypothesis  heH  that  mostly  agrees  with  g  on  the  examples  in  the  training  set.  In  thinking  about  the 
generalizability  of  the  network  it  is  necessary  to  ask:  Does  this  choice,  which  is  based  on  the  behavior  of  the 
examples,  hold  in  general  (Abu*Mostafa,  1989). 

Generalization  can  be  expected  if  the  number  of  examples  is  large  enough.  Without  sufficient  number  of 
examples  the  algorithm  cannot  produce  a  good  hypothesis  With  too  many  examples  the  computational  task 
of  digesting  the  examples  into  a  hypothesis  proves  intractable. 

(3.4)  Adaptive  Pattern  Recognition 

We  will  view  pattern  recognition  as  the  ability  to  map  a  series  of  input  patterns  into  groups  of  output 
patterns  (such  as  classes  of  patterns)  or  as  Pao  (1989)  defines  “a  mapping  from  representation  space  to  interpre¬ 
tation  space”.  The  goal  of  this  research  with  respect  to  pattern  recognition  is  to  determine  if  a  neural  network 
can  be  used  as  the  operator  to  carry  out  this  mapping  for  use  in  fault  diagnosis.  In  the  past,  operators  were 
installed  in  specific  pattern  recognition  systems.  We  will  try  to  develop  a  network  that  can  learn  the  optimal 
operators  through  training.  The  word  adaptive  means  that  the  system  will  be  able  to  deal  with  patterns  that 
are  distorted  with  noise,  be  able  to  modify  classification  rules  in  accordance  with  changing  circumstances,  and 
be  able  to  self-organize  the  connectivity  of  the  network  in  accordance  with  the  structure  of  pattern  data  (Pao, 
1989). 

In  order  to  achieve  an  optimal  operator  we  must  understand  the  use  of  discriminants  in  pattern  recognition. 
A  discriminant  is  a  function  or  an  operator  that,  when  applied  to  a  pattern,  yields  an  output  that  is  either  an 
estimate  of  the  class  membership  of  that  pattern  or  an  estimate  of  the  values  of  one  or  more  of  the  attributes 
of  the  pattern  (Pao,  1989).  Simply  put,  a  discriminant  is  a  type  of  operator  that  produces  a  mapping  from 
pattern  space  into  attribute  space.  The  discriminant  we  are  after  is  one  that  processes  a  complete  pattern  at 
one  time  rather  than  one  feature  at  a  time  The  output  of  the  net  can  be  plotted  as  a  map  of  decision  regions 
created  in  the  multidimensional  space  spanned  by  the  input  variables.  The  outputs  from  a  single  layer  network 
form  half-plane  decision  regions  A  two  layer  net  can  form  many  unbounded  regions  in  the  space  spanned  by 
the  input.  The  output  of  multi-layer  networks  form  complex  convex  regions  based  upon  the  intersection  of  the 
half-planes  formed  by  each  node  in  the  first  layer  of  the  network.  (Lippman,  1987).  These  regions,  bounded  by 
hypersurfaces,  become  our  discriminants  and  patterns  are  classified  in  accordance  whether  they  are  on  one  side 
or  another  of  a  hypersurface  or  a  set  of  hyperplanes  (Pao,  1989)  Using  these  hyperplanes  divides  the  pattern 
space  into  volumes  of  unique  class  membership.  In  this  case  the  classes  are  simple;  normal  or  faulty  operation 
of  the  system. 


(4.0)  RESULTS 

(4.1)  A  Two  Layer  Semilinear  Feed-Forward  Neural  Network:  A  Generalized  Perceptron  Ap¬ 
proach 

The  Autcmatic  Test  and  Control  System  (ATCS)  in  the  ASTF  uses  33  sensor  values  fo  d'agnosis  of  the 
health  of  the  facility.  Eleven  of  those  are  commands  values  from  the  test  cell  engineers  for  changing  the  test 
cell  environment.  The  other  22  are  sensors  that  monitor  the  conditions  of  the  engine  and  the  test  cell.  In  order 
to  catch  all  possible  faults  that  occu.  in  the  ASTF  it  is  necessary  to  monitor  all  22  data  sensors.  The  network 
was  trained  using  a  training  set  of  60  scaled  vectors  composed  of  20  normal  followed  by  40  abnormal  vectors 
made  up  of  22  sensor  values  and  their  desired  outputs.  The  network  was  allowed  to  iterate  1000  times  over 
the  training  set  to  achieve  the  lowest  possible  total  system  error  while  still  achieving  1000  iterations.  Finalized 
connection  weights  for  the  input  nodes  (sensors)  to  the  output  node  ranged  between  -1.61  and  3.78  The  sensors 
with  the  strongest  connection  weights  (highest  absolute  value)  were  a  pressure  sensor  (P0)  v.:*h  a  weight  of 
3.78.  pressure  sensor  PL3H  with  »  weight  of  3  57  and  a  valve  opening  sensor  (LYC1B)  with  a  connection  weight 
value  of  2.02.  The  final  threshold  for  the  input  nodes  was  -0  46.  The  hypcrplanc  formed  by  the  two  layer 
network  is  shown  in  Figure  3. 
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Figure  3  Hyperpiane  Formed  by  a  Two  Lajer  Neural  Network  in  Thiec  Dimensions  (Sensor  PO,  Sensor  PL3H 
and  Sensor  LVC1B) 


In  order  to  interpret  exactly  what  the  network  found,  other  than  by  testing  it  on  new  data  sets,  we  can 
look  at  hew  the  final  weights  converged  and  try  to  decide  if  the  weight  values  are  telling  us  anything  about  the 
system  being  analyzed  Graphing  the  final  weights  m  Figure  4  we  can  see  the  pattern  of  connection  weights 
with  peaks  at  sensors  PO,  PL3H  and  LVClB  As  far  as  this  two  layer  network  is  concerned,  the  majority  of 
relevant  information  as  to  the  health  of  the  system  can  be  gained  from  these  sensors.  In  other  words,  these 
sensor  values  give  the  best  indications  as  to  the  status  of  the  system 

The  two  layer  neural  network  was  tested  on  seven  data  sets  composed  of  ten  vectors  made  up  of  22  sensor 
values.  Two  of  those  sets  were  from  the  training  set  in  order  to  judge  whether  or  not  the  system  was  working 
correctly  The  other  five  were  composed  of  arbitrary  input  vectors  from  the  ASTF  simulator  representing  engine 
test  runs  new  to  the  network.  As  expected,  the  network  computed  a  low  output  node  value  for  the  normal 
vectors  extracted  from  the  training  set  (0  013)  and  a  high  output  node  value  for  the  vectors  representing  faulty 
data  taken  from  the  training  set  (0  999)  This  is  not  surprising  since  we  would  expect  the  network  to  correctly 
identify  input  vectors  it  has  seen  before  The  true  test  of  the  system,  of  course,  is  its  ability  to  make  an  accurate 


Figure  4.  Final  Connection  Weights  foi  a  Two  Layer  Neural  Network 

diagnosis  of  the  health  of  the  system  based  on  some  unknown  input.  When  a  set  of  normal  ASTF  sensor  vectors 
was  presented  to  the  network  the  value  of  the  output  node  value  was  0.014  a  correct  diagnosis  Four  new  data 
sets  composed  of  faulty  sensor  vectors  was  presented  to  the  network  and  the  output  node  achieved  values  of 
0.990,  0.998,  0.936  and  0.990  respectively.  These  output  values  correctly  indicate  that  the  system  is  in  failure 
This  shows  that  the  network  architecture  is  making  accurate  diagnosis  and  generalizing  over  the  space  spanned 
by  the  input  variables.  A  summary  of  the  output  values  for  the  network  tested  on  various  data  sets  is  given  in 
Figure  7 

To  confirm  that  the  network  was  indeed  accurately  classifying  ASTF  sensor  data  from  any  sensor  in  the 
ATCS,  the  network  was  trained  using  a  new  training  set  composed  of  vectors  different  from  those  m  the  original 
training  set.  The  network  was  tested  against  test  vectors  extracted  from  the  training  set  As  expected  the 
system  achieved  high  diagnostic  accuracy  with  a  mean  output  value  of  0.997  for  vectors  representing  faults  and  a 
mean  output  value  of  G.GGG4  foz  vectors  representing  a  normal  test  run  To  test  the  network  for  general iz ability 
over  all  sensors  from  the  ASTF  two  unknown  (not  part  of  the  training  set)  faults  were  presented  to  the  network 
from  unspecified  locations  in  the  test  cell.  The  first  vector  was  an  unknown  representing  a  turbine  trip  at  an 
undisclosed  location  in  the  ASTF  and  the  second  vector  represented  a  compressor  failure  in  an  unspecified  leg 
of  the  test  cell.  The  network  achieved  an  output  value  of  0.998  for  the  first  vector  and  0  992  for  the  second 
vector. 


(4.2)  A  Three  Layer  Semilinear  Feed-Forward  Neural  Network:  A  Multilayer  Perceptron  Ap¬ 
proach 

A  network  was  constructed  using  a  hidden  layer  composed  of  three  and  five  processing  elements.  Tests  were 
run  for  each  type  of  network,  both  on  the  vectors  extracted  from  the  training  set  and  on  the  same  unknown  test 
vectors  as  for  the  two  layer  network.  Since  a  two  layer  network  gave  accurate  responses,  the  three  layer  network 
should  be  thoroughly  scrutinized  to  see  if  it  enhances  the  amount  of  information  gained  from  the  network  Since 
adding  a  layer  adds  computational  complexity  and  adds  to  the  time  necessaiy  to  train  and  test  the  network,  a 
hidden  layer  should  be  able  to  detect  hidden  features  in  the  patterns  of  input  data  not  seen  by  the  two  layer 
network,  and  be  able  to  generalize  over  many  different  test  sets  in  order  to  make  it  economically  worthwhile 

The  first  three  layer  neural  network  used  three  piocessmg  elements  in  the  hidden  layer.  Connection  weights 
from  the  input  nodes  to  the  first  hidden  node  langed  from-1  60  to  0  76  with  highest  (absolute  value)  weights 
from  sensors  P0  (-1  55),  FLoH  (-1  60)  Only  these  two  sensors  had  connection  weight  strenghts  greater  than 
1.0.  Connection  weights  from  the  input  nodes  to  the  second  hidden  node  ranged  from  -1.59  to  0.71  with  greatest 
weight  strengths  from  sensors  P0  (-1  30)  and  PL3H  (-1.59).  Again,  only  these  two  sensors  had  weights  whose 
absolute  value  was  greater  than  1.0.  Connection  weights  to  the  last  hidden  node  ranged  from-1. 27  to  2  50 
with  the  strongest  connections  from  sensors  P0  (2  28),  PL3H  (2.50)  and  LVC2A  (-1.27).  By  adding  a  hidden 
layer  new  information  was  gained  on  diagnosing  the  health  of  the  test  cell,  namely  the  significance  of  data  from 
sensor  LVC2A.  In  the  two  layer  neural  net  the  connection  strength  between  LVC2A  and  the  output  node  was 
high  (1.48),  but  its  relative  importance  in  diagnosing  the  status  of  the  system  was  best  seen  in  studying  the 
connection  weights  between  the  input  layer  and  the  third  hidden  node  The  hyperplanes  formed  by  the  three 
hidden  nodes  can  be  seen  in  Figure  5.  The  addition  of  the  hidden  layer  produced  enough  hyperplanes  to  form 
a  more  complex  region  to  better  classify  the  data 


Figure  5  Hyperplanes  Formed  by  Three  Hidden  Nodes  Graphed  m  Three  Dimensions  (Sensor  PO,  Sensor 
PL3H  and  Sensor  TL1H) 

A  three  layer  neural  netwoik  with  five  hidden  processing  elements  was  also  explored.  For  all  five  hidden 
nodes  the  strongest  connections  between  the  input  layer  and  the  hidden  layer  were  from  nodes  representing 
sensors  P0,  PL3H,  LVClB  and  LVC2A  Based  on  the  finalized  weights  of  the  multilayer  network,  four  sensors 
give  the  strongest  indication  as  to  the  healtn  of  the  ASTF.  A  summary  of  connection  weights  can  be  seen  in 
Figure  6. 

The  three  layer  neural  network  with  three  and  five  hidden  processing  elements  (PEs)  was  tested  on  the 
same  data  sets  as  the  two  layer  network  in  order  to  compare  diagnostic  accuracy  A  summaiy  of  the  results 
is  given  in  Figure  7.  Computing  the  output  for  the  network  with  three  nodes  in  the  middle  layer  using  data 
sets  composed  of  normal  and  faulty  vectors  extracted  from  the  training  set,  resulted  in  an  output  node  value 
of  0  010  and  0.996  respectively.  For  the  test  set  composed  of  new  normal  sensor  data  the  value  on  the  output 
node  was  0  Oil.  For  the  four  faulty  sensor  data  sets  the  network  achieved  the  following  values  0.991,  0.994, 
0.973  and  0.991.  By  comparing  these  values  with  the  outputs  from  the  two  layer  network  it  can  be  seen  that 
diagnostic  accuracy  was  increased  somewhat  in  four  out  of  five  test  cases  by  the  addition  of  the  hidden  layer. 

Testing  the  three  layer  system  with  five  hidden  PEs,  the  network  computed  output  values  of  0  011  for  the 
normal  test  case  extracted  from  the  training  set  and  0  997  for  the  abnormal  test  case  similarly  derived  For 
the  new  unknown  test  cases  the  network  achieved  values  on  the  output  node  of  0.013  for  the  normal  vectors 
and  0.992,  0.99C,  0.944  and  0  992  for  the  fauh  cases  respectively.  This  architecture  did  better  than  the  two 
layer  system  in  the  same  four  out  of  five  test  cases  as  the  system  with  three  hidden  PEs,  but  achieved  a  higher 
diagnostic  accuracy  then  th»  other  three  layer  network  in  only  three  out  of  five  cases. 
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Figure  6.  Final  Connection  Weights  from  Input  Nodes  to  Nodes  of  Hidden  Layer  Based  on  Number  of  Hidden 
Nodes 
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The  results  of  these  tests  prove  that  the  network  is  achieving  accurate  diagnosis  cf  the  health  of  the  ASTF 
over  arbitrary  inputs  and  that  the  network  is  able  to  generalize  over  all  possible  faults  in  this  instance  Based  on 
the  comparisons  in  Figure  7,  it  can  be  seen  that  a  three  layer  neural  network  with  three  hidden  nodes  achieves 
a  slightly  more  accurate  diagnosis  than  a  two  layer  neural  network  The  addition  of  two  more  nodes  m  the 
hidden  layer  only  results  in  a  small  increase  in  performance,  but  significantly  increases  the  time  needed  to  train 
the  network.  Tests  conducted  using  the  two  layer  network  seem  to  indicate  that  data  coming  from  the  ASTF 
sensors  is  linearly  separable.  Based  on  the  increased  run  time  and  greater  computational  complexity  it  appears 
that  a  two  layer  network  is  able  to  diagnose  the  status  of  the  engine  test  cell  with  high  enough  diagnostic 
accuracy  not  to  warrant  the  addition  of  a  middle  layer,  specifically  the  addition  of  five  hidden  nodes.  However, 
m  this  case  a  multilayer  neural  net  proved  useful  in  discovering  sensor  information  that  give  important  insight 
into  monitoring  the  health  of  the  test  cell.  Also  Figure  3  clearly  shows  that  a  two  layer  neural  network  is  unable 
to  seperate  all  classes  of  data,  especially  in  the  presence  of  noise. 

The  strongest  connection  weights  from  the  first  hidden  node  to  the  output  node  were  from  sensors  PL3H 
(-1.60),  PO  (-1.55),  and  TLlH  (-0.87).  Using  these  weignts  to  calculate  the  intercepts  of  the  hyperplane  on  a 
three  dimensional  graph  clearly  shows  that  a  multi-layer  neural  network  with  tliree  hidden  processing  elements 
seperates  data  that  was  non-linearly  separable.  The  ?ud;tion  of  a  middle  layer  adds  diagnostic  accuracy  to  the 
network. 


(5.0)  THE  NETWORKS  ABILITY  TO  HANDLE  NOISE 

A  great  deal  of  criticism  is  given  to  diagnostic  systems  designed  purely  around  simulated  data.  Many 
systems,  neural  networks  among  them,  do  not  perform  as  well  on  real  world  data  as  they  did  on  data  from  a 
simulator.  One  reason  for  this  is  the  noise  that  is  created  when  sampling  real  time  sensor  data.  If  a  neural 
network  is  to  be  used  for  diagnosing  the  status  of  an  engine  test  cell  it  must  be  robust  and  resistant  to  noise. 

Because  of  the  unavailability  cf  actual  engine  test  data  it  was  necessary  to  add  noise  to  the  data  sets 
created  by  the  ASTF  simulator.  The  original  backpropagation  program  was  modified  to  allow  the  user  to  add 
a  certain  percentage  of  noise  to  both  the  training  set  data  and  to  the  test  set  data.  Random  noise  was  added 
to  the  vectors  during  every  pass  through  the  simulated  data  in  order  to  better  resemble  the  noise  that  might 
be  encountered  in  on-line  training  and  testing  of  the  network  Every  time  any  piece  of  data  was  used  by  the 
network  in  either  training  or  testing,  the  random  number  generator  was  re-seedc  i  m  order  to  give  completely 
random  noise.  Tests  of  the  system  were  conducted  with  5%,  10%,  25%  and  35%  noisy  data.  The  algorithm 
used  for  adding  noise  to  previously  used  data  sets  was: 

Noisy  Data  Value  =  (data  value)  +  (uniform  random  number[-l,l])  *  (percent  noise)  (5  0  0) 

A  comparison  of  the  noise  levels  for  all  seven  tests  sets  for  a  two  layer  backpropagation  network  is  given  in 
Figure  8  A  two  layer  backpropagation  neural  network  was  trained  using  a  training  set  modified  to  include  5% 
noise.  Seven  test  data  sets  composed  of  ten  vectors  (two  from  the  training  set  and  five  previously  unknowns) 
were  used  to  test  the  network.  For  the  unknown  normal  vector  set  the  network  computed  an  average  value  of 
0.051.  For  the  four  unknown  fault  vector  sets  the  average  computed  values  were  0.972,  0  993,  0.874  and  0.967. 
All  these  output  values  diagnose  the  condition  of  the  noisy  simulated  data  with  a  high  degree  of  accuracy. 

The  network  was  trained  again  to  include  10%  noise.  The  avciage  output  value  computed  by  the  net  for 
the  normal  test  vector  set  was  0  051.  For  the  four  fault  vector  sets  the  respective  average  output  values  were 
0  973,  0  993,  0  874  and  0.967.  The  performance  of  the  network  degrades  slightly  as  the  level  of  noise  increases 
For  tests  conducted  with  25%  noise  the  average  output  value  of  the  network  was  0  051  for  the  normal  vectors 
and  0  973,  0.992,  0.873  and  0.967  for  the  fault  vectors.  35%  noise  in  the  training  and  testing  sets  resulted  in 
average  output  values  of  0.051  for  the  normal  vector  set  and  0.973,  0.992,  0.872  and  0.966  for  the  fault  sets 
respectively. 
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Figure  8.  A  Two  Layer  Neural  Network  TVained  and  Tested  with  Noisy  Data  Over  100  Iterations 
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Because  a  multilayer  neural  network  spreads  its  knowledge  in  the  form  of  connectioi  ..gilts  over  more 
nodes,  we  would  expect  that  it  is  less  affected  by  noisy  data  and  would  diagnose  the  status  of  the  ASTF  with  a 
higher  degree  of  accuracy.  This  was  indeed  the  case.  The  tests  for  noise  in  a  two  layer  network  were  repeated 
with  a  three  layer  neural  net  with  three  processing  elements  in  the  hidden  layer,  with  results  as  shown  in  Figure 
9. 


For  a  network  trained  and  tested  on  vectors  modified  to  include  5%  noise  the  average  computed  output 
values  were  0.044  for  the  normal  test  vectors  and  0.970,  0  976,  0.927  and  0.968  for  the  fault  vector  sets.  For 
a  network  modified  to  include  10%  noise  in  training  and  testing  the  average  output  values  did  not  change 
from  the  network  trained  and  tested  with  5%  noise.  The  outputs  also  remained  unchanged  for  a  network  that 
included  25%  noise.  Bared  on  this  test  we  can  see  that  a  multilayer  neural  network  is  robust  and  resistant  to 
noise  in  some  instances.  A  slight  degradation  in  diagnostic  confidence  appears  in  training  and  testing  a  network 
to  include  35%  noise.  The  average  output  value  for  the  network  was  0.045  for  the  normal  vector  set  and  0  969, 
0.976,  0.927  and  0.968  for  the  fault  vector  sets. 

By  comparing  the  results  in  Figure  8  and  Figure  9  it  can  be  seen  that  the  three  layer  network  does  better 
on  the  unknown  normal  test  vector  set  and  fault  vector  set  four,  and  much  better  for  the  third  fault  test.  A 
two  layer  network  trained  and  tested  with  35%  noise  for  fault  test  set  three  computes  an  average  output  value 
of  0.872  whereas  a  three  layer  network  computes  an  average  output  value  of  0.927.  It  can  be  seen  from  these 
results  that  a  multilayer  neural  network  is  more  robust  and  resistant  to  noise  than  a  two  layer  generalized 
perception.  If  training  is  to  be  conducted  on-line  in  the  ASTF  a  multilayer  neural  network  should  be  used 
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Figure  9.  A  Three  Layer  Neural  Network  with  Three  Hidden  Nodes  Trained  and  Tested  with  Noisy  Data  Over 
100  Iterations 


(6.0)  THE  NETWORKS  ABILITY  TO 
CLASSIFY  TYPES  OF  FAULTS 

To  make  this  system  more  useful  to  test  cell  engineers  it  would  be  helpful  for  the  network  to  tell  us  what 
type  of  fault  is  occunng. 

The  ASTF  simulator  was  used  to  generate  data  representative  of  four  types  of  faults:  the  ASTF  loses  one 
air  side  compressor  during  a  ramp  in  test  conditions;  the  ASTF  experiences  a  turbine  trip  of  the  only  turbine  in 
use;  the  ASTF  1  >ses  air  side  compressors  while  it  is  idle;  and  some  unknown  fault  that  has  not  been  diagnosed 
by  the  test  cell  engineers  These  faults  along  with  vectors  representing  normal  test  data  are  used  to  compose 
a  training  set  in  the  following  way:  30  normal  vectors,  20  vectors  representing  the  compressor  failure  during 
ramp,  20  vectors  representing  a  turbine  trip,  20  vectors  representing  loss  of  air  side  compressors  while  at  idle, 
and  10  vectors  representing  an  undiagnosed  type  of  fault.  This  combinations  was  picked  by  availability  of  the 
data  and  the  results  discussed  in  3.0. 

Both  a  two  layer  and  three  layer  architecture  were  tested.  In  both  cases  the  network  was  composed  of  22 
input  nodes  representing  the  ASTF  sensors  and  five  output  nodes  representing  normal  (node  0),  compressor 
failure  (node  1),  turbine  trip  (node  2j,  compressor  failure  at  idle  (node  3)  and  undiagnosed  (node  4).  Desired 
outputs  were  added  to  the  training  set  with  a  high  value  (1)  for  the  node  representing  the  type  of  fault  and  a 
low  value  (0)  for  all  other  output  nodes.  To  make  the  data  more  similar  to  real  time  test  data  10%  noise  was 
added.  The  networks  were  trained  for  100  iterations  with  a  learning  rate  of  0.2. 

After  the  network  was  trained  it  was  tested  on  two  vector  sets  from  the  training  set  and  five  unknowns. 
Figure  10  shows  the  results  for  the  two  layer  network.  For  the  test  set  composed  of  normal  vectors  the  value  of 
node  0  was  high  (0.371)  with  all  other  nodes  being  relatively  low.  For  the  first  fault  test  set  the  value  of  node 
1  representing  a  compressor  failure  was  high  (0.601)  with  all  other  output  nodes  being  relatively  low.  Likewise 
for  fault  teat  two  the  value  for  node  1  was  high  (0.914)  while  all  others  remained  low.  Fault  test  three  contained 
data  taken  from  the  ASTF  during  a  turbine  trip  and  the  value  for  node  2  was  high  (0.479).  Fault  test  four 
(compressor  snap  at  idle)  had  a  high  value  for  node  3  (0.826)  and  a  reasonably  high  value  for  node  0  (0.312). 
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A  three  layer  network  with  three  hidden  processing  units  was  tested  for  its  ability  to  classify  types  of  faults 
After  the  same  training  configu'  ation  as  the  two  layer  network  the  net  was  tested  against  the  same  five  test 
cases.  The  three  layer  network  did  poorly  at  diagnosing  normal  test  vectors  for  both  the  unknown  normal  and 
normal  data  taken  from  the  training  set.  In  both  cases  the  network  diagnosed  the  data  as  being  representative 
of  the  undiagnosed  fault.  However,  the  multilayer  network  did  very  well  in  diagnosing  some  types  of  faults.  For 
fault  test  one  the  value  of  node  1  was  high  (0.535)  For  fault  test  two  (another  compressor  failure)  the  value 
of  node  1  was  0.876.  For  the  turbine  failure  fault  test  three  the  value  of  node  2  was  0.848  and  for  fault  test 
four  the  value  of  node  three  was  high  (0.864)  with  all  other  output  nodes  being  relatively  low.  These  results 
are  summarized  in  Figure  11 

Based  on  these  results  it  would  seem  that  the  multilayer  neural  network  can  diagnose  certain  types  of  faults 
with  high  degrees  of  accuracy.  However  this  network  architecture  is  not  able  to  differentiate  between  normal 
data  and  unknowns.  While  a  two  layer  network  does  not  give  output  values  with  as  high  a  confidence  as  the 
multilayer  network,  it  is  able  to  diagnose  the  status  of  the  system  with  accuracy  and  with  no  error  duiing  these 
tests 
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(7.0)  CONCLUSIONS  AND  RECOMMENDATIONS 

This  research  has  shown  that  a  neural  network  can  be  used  to  differentiate  between  patterns  of  data 
representing  a  normal  test  run  of  the  Aeropropulsion  Systems  Test  Facility  and  sensor  data  coming  from 
a  test  cell  that  is  in  failure  A  network  was  also  built  to  classify  the  type  of  fault  that  is  occunng  For 
simulated  ASTF  data  a  near  optimal  diagnosis  can  be  achieved  using  a  two  layer  neural  network  (generalized 
perccptron).  However,  for  simulated  data  with  noise  such  as  would  occur  in  actual  engine  test  cell  data,  a 
multilayer  neural  network  with  three  hidden  processing  elements  yields  a  more  accurate  diagnosis  The  neural 
network  architecture  described  in  this  report  diagnoses  the  health  of  the  ASTF  with  a  high  degree  of  accuracy 
after  being  exposed  to  an  input  vector  representing  sensor  data  from  only  one  time  slice  (0.2  sec)  and  an  even 
higher  accuracy  after  averaging  ten  time  slices,  or  two  seconds,  worth  of  data. 

In  addition,  by  monitoring  the  converging  weights  of  the  input  processing  elements  it  was  seen  that  the 
most  relevant  information  used  by  the  22  input  nod*  network  comes  from  four  sensors,  P0,  PL3H,  LVC1B  and 
LVC2A  based  on  connection  strengths.  Graphing  t»  node  connection  weights  and  the  total  system  error  for 
various  network  parameters  shows  whether  the  weights  oscillate  while  approaching  error  minima. 

,  This  research  has  demonstrated  the  usefulness  of  neural  networks  for  fault  monitoring  and  diagnosis. 

Further  research  and  implementation  of  neural  network  based  systems  will  give  test  personnel  a  new  tool  in 
I  the  anlysis  of  critical  test  data.  This  system  will  help  take  neural  network  technology  out  of  the  laboratory 

\  anc*  kelp  create  commercial  systems  for  the  analysis  of  complex  systems.  The  most  significant  potential  benefit 

j  *ke  system  which  has  been  developed  is  the  improved  safety  of  test  cell  personnel  by  rapid  detection  of 

!  faulty  conditions.  This  will  be  of  extreme  importance  as  testing  begins  on  more  advanced  systems  such  as  the 

;  National  Aerospace  Plane  and  the  Advanced  Tactical  Fighter. 
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SUMMARY 

Hie  application  of  Ada  and  an  object-oriented  approach  to  the  design  and  construction  of  advanced  defence  systems  are  both 
attracting  increasing  attenuon.  The  Ada  language  contains  some  support  for  object-oriented  programming  but  has  some  notable 
deficiencies.  This  paper  shows  how  to  overcome  these  deficiences  by  providing  a  library  for  object-oriented  development  in  Ada 
( OODA  )  which  contains  facilities  to  create  and  manipulate  objects  and  provides  support  for  more  general  relationships  between 
objects 

One  of  the  initial  applications  of  this  bbrary  has  been  to  design  a  framework  for  integrating  KBS  with  control  system  simulations 
comprising  mixed  continuous  and  discrete  time  elements.  Using  this  framework  it  is  possible  to  study  the  interactions  between  a 
Knowledge  Based  controller  and  the  other  more  conventional  elements  of  the  closed  loop  to  any  level  of  detail  required. 

1.  INTRODUCTION 

The  object-oriented  approach  to  the  development  of  complex  systems  is  attracting  increasing  support  for  its  promise  of  improved 
maintainability  and  reusability  through  modelling  the  software  as  a  collection  of  interacting  objects.  There  are  a  number  of 
object-oriented  languages  ( Simula,  Smalltalk,  C++  etc.  )  which  all  embody  different  versions  of  the  four  basic  concepts  of 
object-oriented  programming,  namely: 

Encapsulation'  the  property  that  a  software  object  is  a  completely  self-contained  entity,  possessing  all  the  data  and  code 
needed  to  perform  a  set  of  standard  operations  which  form  its  only  interface  to  other  objects. 

Data  abstraction:  the  ability  to  treat  the  definition  of  an  single  object  as  an  abstract  entity  which  can  then  be  used  to  define 
many  further  objects  of  the  same  type,  all  with  independent  data. 

Inheritance:  the  ability  to  define  new  types  of  object  which  have  some  of  the  properties  of  old  types,  and  some  new  pro¬ 
perties  of  their  own.  These  new  types  can  be  regarded  as  specialised  variants  of  the  old.  Multiple  Inheritance  allows  the 
properties  of  more  than  one  parent  to  be  combined. 

Dynamic  binding:  a  facility  of  many  object-oriented  languages  which  allows  flexibility  in  the  specification  of  object  opera¬ 
tions.  A  message  can  be  sent  to  an  object  requesting  it  to  perform  some  action,  without  having  to  specify  the  type  of  'his 
object  until  run  time. 

It  is  important  to  emphasise,  however,  that  the  use  of  object-oriented  techniques  is  not  dependent  on  any  particular  programming 
language.  Rather  it  is  an  approach  to  organising  and  planning  computer  programs  which  can  be  applied  to  a  greater  or  lesser 
extent  in  all  software  development.  The  standard  language  for  defence  systems,  Ada,  contains  support  for  some  of  the  basic 
concepts  of  object-oriented  programming.  Encapsulation  can  be  achieved  by  the  use  of  packages,  and  data  abstraction  by 
defining  abstract  data  types,  instances  of  which  can  be  replicated  (  Booch  (1) ).  Inheritance  and  dynamic  binding  are  not  readily 
emulated,  however,  and  other  features,  which  aid  the  construction  of  a  world  of  interacting  objects,  are  entirely  absent. 

The  OODA  library  described  here  remedies  these  deficiencies  by  providing  support  for  a  range  of  features  usually  regarded  as 
essential  for  object-oriented  programming,  while  staying  entirely  within  the  Ada  language  standard.  No  translators  or  other  pre¬ 
processors  are  needed,  in  contrast  with  some  other  approaches  to  this  problem  (  Simonian  and  Crone  (2);  Kovarik  and  Nies  (3) ). 
The  library  also  supports  the  definition  of  more  advanced  structures  and  object  relationships  and  has  been  designed  to  be  as 
efficient  as  possible  in  execution  to  make  it  suitable  for  use  in  real-time  systems.  In  this  paper,  we  describe  the  OODA  library 
and  initial  applications  of  it  to  the  construction  of  knowledge  based  systems  and  to  a  framework  for  mixed  continuous/discrete 
simulations,  capable  of  interacting  with  these  knowledge  bases. 

2.  THE  ADA  OBJECT-ORIENTED  LIBRARY 

Within  the  library,  the  basic  mechanism  used  to  encapsulate  the  operations  and  data  for  an  object  is  the  Ada  package  structure. 
In  practice,  provision  has  to  be  made  to  replicate  objects  of  one  type  any  number  of  times.  To  achieve  this,  each  package 
defines  a  class  of  objects,  and  the  individual  instances  of  this  class  are  created  as  needed,  and  destroyed  when  needed  no  longer. 
The  attributes,  or  data,  for  these  independent  objects  are  all  stored  within  the  class  package  as  an  array  of  reusable  records.  The 
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maximum  size  of  the  set  of  instances  which  can  exist  at  any  one  time  is  determined  at  the  start  of  a  run,  when  this  class  set  is 
initialised.  While  not  being  as  flexible  as  a  system  which  allows  objects  to  be  replicated  ad  libitum,  this  approach  gives  fast  and 
efficient  access  to  the  attributes  of  either  an  individual  object  or  the  set  of  such  objects.  It  should  be  noted  that  the  strong  typing 
capability  of  Ada  is  used  to  ensure  that  all  references  to  objects  of  a  particular  class  are  made  via  a  distinct  type  Such  a  struc¬ 
ture  allows  several  categories  of  error  in  the  use  of  classes  to  be  identified  at  compile-time. 

2.1  Accessing  Objects 

An  object  is  uniquely  identified  by  its  name  and  the  class  to  which  it  belongs.  An  object  can  be  referred  to  by  name,  but,  in 
order  to  improve  efficiency,  the  principal  method  for  accessing  an  object  is  via  a  privately-typed  reference.  The  reference  is 
assigned  when  the  object  is  created,  and  includes  both  a  unique  identity  for  the  object,  and  the  location  of  the  object’s  attributes 
within  its  class.  The  type  of  the  reference  is  determined  by  the  class  of  the  object.  The  following  fragment  of  Ada  code  from  a 
typical  defence  application  shows  the  creation  of  an  object  named  "Harrierr',  of  class  Fighter,  accessed  by  a  reference  HI: 

HI  •:  Fighter_type;  -  define  the  type  of  the  reference 

Createf  HI,  name  ->  "Harrierl"  );  —  create  the  object 

Refuelf  HI );  -  access  to  it  by  reference 

The  alternative  method  of  access  is  by  giving  the  name  of  the  object  and  using  a  specially  defined  operation,  named  after  the 
class  concerned,  to  return  the  appropriate  reference,  e.g.: 


Refuelf  Fighterf  "Harrierr'  ));  -  access  by  name 


2.2  Inheritance  and  Multiple  Inheritance 

Inheritance  is  the  method  used  in  object-oriented  languages  to  allow  new  classes  of  object  to  be  constructed  by  reusing  previ¬ 
ously  defined  classes.  A  new  object  class  can  inherit  the  properties  of  an  old  one,  adding  new  properties  of  its  own  and  thus 
becoming  a  specialised  variant  of  the  old  class.  This  can  be  thought  of  as  constituting  an  ”is-a”  relationship  between  the  two 
classes,  e.g.  a  Fighter  "is-a"  Moving-object.  The  method  used  to  support  inhentance  in  this  library  is  a  form  of  construction,  as 
proposed  by  Taenzer  et  al.  (4)  to  allow  more  effective  class  reuse  in  Objective-C.  When  a  new  (child)  object  wishes  to  inherit 
from  an  old  (parent)  class,  it  creates  a  new  instance  of  the  parent  class  wuh  the  same  name  as  itself,  e  g.. 


MovObj  :  Moving_Object_type; 
Create(  MovObj,  kin_of  =>  Self ); 


-  define  the  reference  for  the  parent. 

-  create  the  inherited  parent  object, 

-  "kin_of"  denotes  inheritance,  "Self”  the  child 


The  operations  of  the  parent  are  then  available  for  use  within  the  child  object  by  reference,  or  external  to  it  by  name  or  relation¬ 
ship.  Multiple  inheritance  is  readily  achieved  since  parent  instances  can  be  constructed  from  any  number  of  classes.  Where  the 
parents  are  themselves  derived  from  a  common  ancestor  class  ( i.e.  repeated  inheritance,  see  Meyer  (5) ),  the  library  will  create 
only  one  instance  of  this  class. 

2.3  Component  Hierarchies 

Another  way  in  which  complex  objects  are  created  is  to  build  them  up  out  of  simpler  component  objects  or  parts.  The  com¬ 
ponent  objects  are  best  thought  of  as  being  contained  within  the  complex  object,  though  in  practice  they  have  a  separate 
existence  within  their  own  class,  and  can  be  accessed  individually  when  required,  e.g.  to  initialise  them.  These  components  can 
themselves  be  made  up  of  others,  resulting  in  a  hierarchy  of  component  objects. 

Componency  is  more  flexible  than  inheritance  as  a  means  of  constructing  complex  objects  since  a  class  can  be  inherited  only 
once,  but  it  is  perfectly  possible  to  have  more  than  one  component  from  the  same  class.  Each  component  is  distinguished  by  a 
part-name,  e.g.  "Left_Wheer,  ''Right_Wheer  and  the  name  given  to  a  object  component  is  made  up  of  the  name  of  its  owner 
appended  to  its  part  name,  e.g.  "Hanierl.Left_Wheel".  The  components  are  accessed  within  the  owner  object  by  reference  and 
outside  it  by  name  or  relationship.  For  example,  within  the  owner,  we  can  create  components  and  access  them  thus: 
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Wl,  W2  :  Wheel,  type; 

■■define  references  to  component 
Create(  Wl,  name  =>  "Left_Wheel’',  pait_of  =>  Self ); 
Create(  W2,  name  =>  "Right_Wheel",  pait_of  =>  Self ); 
-create  components,  "part_or  denotes  componency. 


Outside  the  owner,  name  or  relationship  are  used; 

Inflate(  Wheel(  "Hanierl.Right_Wheel" ),  Pressure  ); 

-access  by  name. 

Inflate(  Wheel{  "Right_Wheel",  part_of  =>  HI ),  Pressure  ); 

-equivalent  (but  faster)  access  by  part  name. 

2.4  Sub  Sets  and  Set  Operations 

Inheritance  and  componency  arc  both  permanent  relationships,  holding  true  for  the  lifetime  of  an  object.  There  are,  however, 
many  problems  in  which  it  is  desirable  to  form  temporary  associations  between  groups  of  similar  objects.  The  OODA  library 
provides  this  facility  by  allowing  the  formation  of  Sub-Sets  which  can  be  populated  by  any  selection  of  the  objects  in  that  class. 
Objects  can  be  added  or  removed  individually,  or  a  new  sub-set  can  be  made  up  of  a  logical  combination  of  old  sub-sets,  or 
from  a  general  object  specification. 

Sub-sets  are  particularly  useful  in  the  construction  of  plans  for  groups  of  objects,  the  members  of  which  can  be  altered  as  the 
plan  matures.  A  complete  sub-set  of  objects  from  one  class  can  be  inherited  by  a  single  object  from  another  class,  allowing 
one-to-many  mappings  to  be  created  between  objects  of  different  classes.  For  example,  an  object  describing  a  meeting  could 
inhent  a  sub-set  of  people  who  are  booked  to  attend  it  (  a  Meeting  "is-a"  Set-of-people ).  A  single  object  can  be  included  in 
several  sub-sets  at  the  same  time,  allowing  many-to-many  mappings  to  be  constructed.  In  this  example,  one  person  could  be 
booked  to  attend  many  meetings.  Using  the  library,  it  is  possible  to  find  out  who  is  booked  for  any  two  meetings  by  forming  the 
intersection  between  their  sub-sets. 

A  further  enhancement  to  aid  the  manipulanon  of  sets  and  sub-sets  of  objects  is  the  ability  to  define  Set  Operations.  Rather 
than  operating  on  one  instance  at  a  time,  a  set  operation  can  perform  the  same  operation  on  the  members  of  the  whole  set  of 
objects  in  a  class,  or  any  of  its  sub-sets.  This  can  considerably  clarify  the  code,  and  improve  run-time  performance  by  reducing 
the  number  of  procedure  calls  needed  to  perform  a  given  task.  Set  operations  are  particularly  useful  when  constructing  general 
facilities,  such  as  inference  engines,  for  which  access  to  the  world  of  objects  independently  of  any  specific  application  software 
is  desirable. 

2 .5  Additional  Features 

1.  A  template,  containing  a  number  of  pre-defined  operations,  is  provided  to  simplify  the  creation  of  class  packages.  The 
specifications  and  bodies  of  class  packages  are  compiled  separately  to  minimise  recompilation  after  changes. 

2.  Different  classes  can  use  the  same  names  for  operations.  The  type  of  the  object  reference  is  used  to  select  the  right  one  at 
compile  time. 

3.  Dynamic  binding  can  be  simulated  for  specific  operations  by  including  additional  Ada  procedures  to  select  the  correct 
operation  at  run  time. 

4.  An  interactive  tracing  and  diagnostic  package  is  included,  allowing  operations  on  individual  objects  to  be  logged,  and  their 
attributes  displayed. 

5.  Objects  have  extended  visibility  -  they  can  be  accessed  by  name  from  any  part  of  a  large  Ada  task.  It  is  possible,  however, 
to  prevent  critical  object  operations  from  being  accessed  from  unauthorised  parts  of  a  task  by  use  of  the  optional  access 
protection  facility. 

3.  APPLICATION  TO  KBS 

This  illustrates  the  use  of  the  OODA  library  to  construct  a  set  of  interacting  object  classes  which  together  constitute  an  inference 
engine  ( see  Figure  1 ).  Three  classes  are  defined;  Facts,  Rules  and  Rulcbases. 

The  first  class  defines  objects  which  are  Facts.  Each  fact  has  a  name  (  as  do  all  objects )  and  an  attribute  describing  the  current 
state  of  knowledge  about  that  fact,  which  can  be  True,  False  or  Unknown.  Facts  can  be  manipulated  both  by  inference  engines, 
and  by  other  objects  within  the  overall  application  program,  allowing  this  class  to  act  as  a  blackboard  system. 
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The  second  class  defines  objects  which  are  Rules.  These  have  attributes  describing  the  antecedent  facts,  and  how  to  combine 
them  using  <and,  or,  not>  functions  to  evaluate  the  rale.  In  addition,  the  attributes  contain  u  list  of  consequent  actions  to  per¬ 
form  if  the  rule  fires  and  an  explanation  of  the  significance  of  the  rale.  The  operations  for  rale  class  include  Evaluate,  which 
inspects  the  antecedents  to  see  whether  or  not  the  rule  is  capable  of  firing,  and  Fire  which  fires  it  and  perforins  the  consequent 
actions,  as  well  as  several  operations  needed  to  define  the  rales. 

The  final  class  defines  objects  which  are  Rulebases.  These  are  responsible  for  the  overall  control  of  the  infercncing  procedure 
for  a  group  of  rules.  Each  rulebase  object  inherits  a  sub-set  of  rules  (  a  Rulebase  "isa"  set-of-Rules  ,,  has  attributes  describing 
the  type  of  infercncing  to  use  with  these  rales  and  a  reference  to  a  goal  fact  and  a  constraint  rulebase.  Its  principal  operation  is 
Invoke,  used  to  start  ihe  appropriate  inference  procedure  for  a  rulebase.  While  infercncing  on  one  rulebase,  another  one  may  be 
invoked,  either  as  a  consequence  of  one  of  the  rales  firing,  or  as  a  consequence  of  the  operation  of  the  inference  engine  itself. 
Another  possible  consequence  of  a  rale  firing  is  to  Queue  a  discrete  event  in  the  simulation  framework  described  in  the  next  sec¬ 
tion. 

Rather  than  create  a  new  language  in  which  to  define  the  rales,  it  was  felt  to  be  preferable  to  keep  the  definition  within  the  syn¬ 
tax  of  Ada,  since  it  is  a  well  defined  standard,  supported  by  a  range  of  software  engineering  tools.  The  basic  form  of  a  rale 
appears  thus: 

Start_Rule(  <rale  references,  <ralename>  ); 

IF  Establishedf  <antecedents>  )  THEN 
<consequents>; 

END  IF; 

£nd_of(  crule  references ); 

The  antecedents  consist  of  a  list  of  factnames  ( in  quotes )  separated  by  <and,  or,  nots  operators  as  appropriate.  The  conse¬ 
quents  comprise  a  list  of  operations  to  be  called  to  redefine  facts,  Explain  the  rale,  or  Invoke  another  rulebase.  The  rulebase 
itself  is  defined  similarly: 

Create(  crulebase  references,  crulebase  names, 

cinfercnce  methods  ), 

Rule  definitions 

End_of(  <rulebase  references  ); 

These  Ada  definitions  are  compiled  with  the  class  packages  for  Rule  and  Rulebase,  which  overload  the  standard  Ada  operations 
AND,  OR  and  NOT  called  within  the  definmons.  The  definitions  are  then  executed  once  to  initialise  the  objects  before  inferenc- 
mg  starts. 

4.  APPLICATION  TO  SIMULATION 

The  other  mam  application  of  the  OODA  library  so  far  has  been  to  simulation.  Below  we  describe  a  mixed  continuous/discrete 
simulauon  system,  constructed  using  OODA,  which  can  run  multiple  continuous  models  under  the  control  of  a  discrete  event 
scheduler,  and  allows  the  models  to  interact  with  knowledge  bases.  Three  groups  of  object  classes  arc  involved  in  this: 

a)  A  class  of  states,  which  are  the  results  of  an  integration. 

b)  Continuous  or  discrete  models,  which  belong  to  various  classes,  some  supplied  by  the  user,  o'hers  provided  in  a  support 
library. 

c)  Integration  control  objects,  inherited  oy  continuous  models,  responsible  for  selecting  and  applying  the  integration  method 
for  that  model. 

These  three  types  of  object  interact  to  provide  the  simulation  facilities. 

States  are  relatively  straightforward  objects,  see  Figure  2.  They  have  operations  for  setting  the  differential  of  an  individual  state, 
and  reading  the  back  the  value  of  that  state;  these  operations  are  used  within  tne  models.  The  operations  which  can  integrate  the 
differentials  to  produce  the  next  values  of  the  states  are  implemented  as  Set  Operations,  which  act  on  the  whole  set  of  states,  or 
any  sub-set  of  it.  These  operations  are  invoked  under  control  of  the  event  scheduler,  transparently  to  the  models  which  use  each 
state. 
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Models  contain  the  equations  ol  motion  being  simulated,  ami  any  calls  to  the  event  scheduler  to  explicitly  schedule  discrete 
events.  They  are  generally  coimmna  objects,  built  up  of  components  which  are  sub-models  provided  by  the  user,  or  hirers  and 
transfer  functions  exunaed  from  a  library.  Both  a  mode)  and  its  .sub-models  would  generally  contain  states  as  components. 

Integration  Controllers  are  inherited  by  models  and  contain  the  information  needed  to  integrate  die  model,  including  the 
integration  algorithm  to  be  used  (  rectangular  and  Runge-Kutta  2nd  and  4th  order  methods  me  current!}'  provided  )  and  the  max¬ 
imum  time  step.  They  hi  their  turn  inherit  a  clock  object  to  recoid  the  progress  of  the  integration,  and  a  Sub  set  of  States  con¬ 
taining  all  the  states  for  the  model.  The  inheritance  structure  is  summarised  in  Figure  3.  This  use  of  sub-sets  of  states  allows 
different  models  to  be  integrated  independently  of  each  other,  with  different  algorithms  and  update  rates. 

As  well  as  integrating  a  model  to  produce  time  responses,  it  is  possible  to  perform  a  small  perturbation  analysis  of  it  to  extract 
its  differential  equations  in  linear  form.  Various  forms  of  s-plane  linear  analysis  techniques  can  then  be  applied  to  predict,  for 
example,  the  stability  and  frequency  responses  of  a  closed  loop  control  system.  So  far,  stability  analysis  has  been  implemented. 

Sampled  data  systems,  involving  difference  equations  rather  than  differential  equations,  can  also  be  accommodated  within  the 
same  framework.  This  allows  systems  consisting  of  a  sampled-data  digital  control  system  controlling  a  continuous  plant  to  be 
simulated.  Linearisation  can  also  be  performed  on  these  sampled  data  systems,  opening  the  way  to  z-plane  analysis  of  the 
overall  control  loop. 

These  various  types  of  model  can  interact  with  a  Knowledge  Based  System  by  executing  discrete  events  which  can  Invoke  the 
tnferencing  operation  of  a  Rulebase.  The  Rulebase  in  its  turn  can  fire  rules  whose  consequents  are  to  Queue  the  operation  of 
further  discrete  events.  Facts  may  be  altered  and  inspected  by  the  KBS  and  by  the  simulation.  This  allows  the  construcuon  of 
control  loops  incorporating  knowledge  based  systems  as  one  of  their  elements.  The  monitoring  facilities  in  the  simulation  frame¬ 
work  and  the  diagnostic  tracing  capability  of  the  OODA  library  itself  can  both  be  used  to  examine  the  detailed  interaction 
between  the  various  elements  of  the  control  system. 

Neural  Network  simulations  have  also  been  constructed  using  the  OODA  library,  and  work  is  currently  under  way  to  investigate 
the  use  of  neural  networks  to  detect  faults  in  control  systems.  The  networks  are  trained  using  the  stored  responses  of  the  control 
system  model  to  various  inputs  and  failure  cases.  The  discrete  event  controller  finds  an  additional  use  in  supervising  the  pro¬ 
gress  of  this  training. 

5.  CONCLUDING  REMARKS 

The  object-oriented  view  of  software  is  rapidly  gaining  support  as  a  flexible,  maintainable  and  intuitive  method  for  structuring 
complex  systems.  This  paper  has  described  a  new  approach  to  the  problem  of  development  and  maintenance  of  such  software: 
the  OODA  library  retains  all  the  software  engineering  properties  built  into  the  Ada  language  and  its  support  environments,  while 
providing  a  rich  set  of  facilities  for  use  with  the  obiect-oriented  paradigm.  These  include  support  for  temporary  associations 
between  groups  of  similar  objects  (Sub-sets)  and  the  definition  of  operations  which  act  on  all  the  members  of  a  sub-set  (Set 
operations). 

Although  intended  primarily  for  use  in  knowl-uge-based  systems  and  simulation,  the  library  is  not  specialised  and  can  be  used  to 
support  any  application  for  which  Ada  is  appropriate.  Initial  applications  have  included  an  framework  for  continuous  and 
discrete  simulations,  inference  engines  for  knowledge  based  systems  and  a  method  by  which  the  two  can  interact  when  model¬ 
ling  control  systems  which  incorporate  a  KBS.  Neural  network  simulation  has  also  been  successfully  undertaken. 

REFERENCES 

1.  Booch  G,  Software  Components  with  Ada, 

Benjamin/Cummings,  Menlo  Park,  California,  1987. 

2.  Simonian  R,  Crone  M,  InnovAda:  True  Object-Oriented 
Programming  in  Ada,  Journal  of  Object  Oriented 
Programming,  Vol.  I.  No.  4, 1988,  pp  14-21. 

3.  Kovarik  V  J  Jr.,  Nies  S,  Supporting  Object-Oriented 
Programming  Within  Ada:  Extending  the  Paradigm, 

AIDA-88,  Fairfax,  VA,  November  1988,  paper  13. 

4.  Taenzer  D,  Garni  M,  Podar  S,  Problems  in  Object-Oriented 
Software  Reuse,  ECOOP  89  Proceedings,  Cambridge 
University  Press,  July  1989,  pp  25-38. 

5.  Meyer  B.1988,  Object  Oriented  Software  Construction, 

Prentice-Hall,  New  York,  1988. 


34-6 


Rules 


operation : 
Invoke 


operations: 
Forward  chain 
Backward  chain 
Hypothesise 


operations: 

Evaluate 

Fire 


Facts 


o 

o 

o 

o 

o 


Figure  1.  Object  Class  Relationships  within  the  KBS  System 
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Figure  2  Object  and  Set  Operations  for  the  class  of  States 


Model  Integration  Control  Clock 


Figure  3.  Inheritance  Relationships  for  Continuous  Simulation. 


LA  PROBLEMATIQUE  MULTI-AGENTS  DANS  LE  COPILOTE  ELECTRONIQUE 
(APPLICATION  DES  SYSTEMES  A  BASE  DE  CONNAISSANCES  AU  GUIDAGE  -  PILOi  AGE) 

par 

Antoine  Gilles 
Dassault-Aviation 
92210  Sajit  Clout 
France 


RESUME 


La  conduite  d'un  avion  de  combat  est  une  tache  difficile  du  type  "controle 
de  processus  complexe"  qui  necessite  tout  particulierement  une  maitrise  de  la 
complexite  temporelle  de  la  mission.  Dassault-Aviation  sous  I'egide  de  la 
DRET(”  (contrat  86-34407)  s'oriente  actuellement  vers  la  realisation  d'un  systeme 
interactif  d'aide  au  pilote  appeie  "Copilote  Electronique".  Ce  projet  complexe 
met  en  oeuvre  plusieurs  entites  expertes.  L'existence  de  telles  entites  pose  des 
problemes  de  gestion  de  ressources  de  communication  et  de  cooperation.  Un 
superviseur  entre  les  differentes  entites  expertes  du  systeme  embarque,  et  une 
architecture  logicielle  adaptee  a  I'arbitrage  et  a  la  communication  de  ces  entites 
est  done  necessaire.  Cet  expose  situe  les  travaux  que  nous  avons  effectues 
dans  ce  domaine  et  les  resultats  obtenus.  II  indique  les  perspectives  que  nous 
envisageons  en  fonction  des  difficultes  rencontrees. 


LA  PROBLEMATIQUE  MULTI-AGENTS  DANS  LE 
COPILOTE  ELECTRONIQUE 


La  conduite  d'un  systeme  embarque  lors  de  missions  complexes  a  toujours 
pose  des  problemes  de  charge  de  travail  pour  un  pilote  seul.  Ces  problems* 
ont  ete  generalement  resolus  par  ('application  de  regies  strictes  de  conduite  de 
la  mission. 

A  I'avenir  les  capteurs  et  les  moyens  de  communication  fourniront  des 
informations  de  plus  en  plus  nombreuses,  donnant  au  pilote  la  possibility  d'une 
veritable  gestion  de  !a  mission.  Cependant,  meme  si  le  pilote  se  trouve  dechar¬ 
ge  des  taches  courantes  et  repetitives,  la  gestion  d'une  mission  complexe  ris¬ 
que  de  constitue*-  une  tache  trop  lourde.  L' Intelligence  Artificielie  pouvant 
proposer  des  reponses  a  ces  problemes,  Dassault-Aviation  travaille  au  develop- 
pement  d'un  systeme  d'aide  au  Pilote  (projet  “Copilote  Electronique”)  et  au  lan- 
cement  des  recherches  necessaires. 
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Afin  de  pouvoir  decomposer  le  probleme  complexe  de  I'assistance  au  pilote 
d'un  systeme  embarque  en  sous  problemes  plus  simples  et  mieux  identifies, 
une  approche  resolument  "multi-experts"  a  ete  adoptee  pour  le  projet.  Cette 
approche  permet  d'elaborer  des  entites  expertes  speciallsees,  capables  de  rai- 
sonner  sur  un  domaine  d'expertise  limite  et  mieux  defini. 

La  validite  des  decisions  proposees  par  un  tel  systeme  doit  toujours  etre 
envisagee  dans  le  contexte  global  de  la  mission.  De  ce  fait  une  assistance  au 
pilote  dans  sop  ensemble  ne  peut  etre  vuo  comme  une  collection  de  systemes 
experts  independants  mais  plutot  comme  un  ensemble  d'agents  intelligents  col- 
laborant  a  I'elaboration  d'une  decision.  Une  telle  collaboration  pose  de  nom- 
breux  problemes  de  gestion  temporelle  des  ressources,  de  communication 
d'information  entre  agents,  et  d'arbitrage  entre  decisions. 

Le  theme  des  univers  multi-agents  prend  une  importance  grandissante  en 
IA,  les  agents  pouvant  etre  aussi  bien  des  entites  physiques  (robots),  que  logi- 
ciels  (systemes  experts)  ;  ils  doivent  cooperer  pour  solutionner  un  certain  pro¬ 
bleme.  Ce  theme  pose  des  problemes  nouveaux  de  representation  des 
connaissances,  de  raisonnement,  d'architecture  des  systemes.  II  s'inscrit  dans 
le  courant  general  de  I'lA  distribute,  concept  appele  a  un  developpement  eco- 
nomique  majeur. 

Une  approche  multi-agents,  consistant  a  faire  cooperer  un  ensemble  de 
systemes  de  traitement  d'informations  et  de  bases  de  connaissances  pour  par- 
venir  a  resoudre  un  probleme  complexe,  s'avere  fructueuse  a  plusieurs  titres  : 

•  du  fait  de  la  nature  distribute  de  certaines  applications  :  plusieurs  acteurs 
peuvent  intervenir;  plusieurs  sous-taches  peuvent  etre  isolees;  les  donnees 
provenant  de  divers  endroits  geographiques  et  de  divers  capteurs  doivent 
etre  fusionnees;  etc. 

•  du  fait  de  la  simplification  qu'il  en  resulte  dans  la  resolution  d'un  probleme 
(technique  IA  de  la  reduction  de  problemes). 

II  existe  plusieurs  facons  d'implanter  un  systeme  multi-agents,  notamment 
les  langages  d'acteurs  et  le  modele  de  blackboard,  ce  dernier  assurant  une  plus 
grande  variete  de  structures  de  controle.  Les  langages  "orientes  objets"  sont 
tres  interessants  pour  realiser  pratiquement  un  tel  modele. 

Quelles  que  soient  les  implantations  retenues,  I'existence  et  le  controle  des 
aiverses  entites  expertes  necessitent  la  presence  d'un  superviseur  qui  puisse 
gerer  dans  le  temps  les  traitements  des  entites  informatiques.  Nous  avons  ela- 
bore  un  premier  module  de  superviseur  dans  la  maquette  actuelle.  Nous  le 
decrivons  ci-apres. 

Maquettaqe 

La  cooperation  entre  differentes  sources  de  connaissance  est  I'un  des  pro- 
blemes  majeurs  auxquels  I'lntelligence  Artificielle  est  confrontee  des  lors  qu'elle 
s'attaque  &  des  problemes  de  grande  ampleur. 

Pour  ©valuer  la  faisabilite  d'un  Copilote  Electronique,  base  sur  des  techni¬ 
ques  d'intelligence  artificielle,  une  maquette  s'averait  indispensable.  Une 
maquette  a  done  ete  realisee.  Elle  constitue  un  outil  de  developpement  et  deva¬ 
luation  des  aides  a  la  decision  expertes  proposees  par  ie  Copilote  Electronique. 
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Cette  maquette  s'appuie  sur  one  simulation  simplifiee  mais  realiste  de  I'environ- 

nement  operationnel  permettant  de  “jouer”  divers  scenarios  de  mission. 

1.  Une  partie  du  logiciel  realise  les  fonctions  d'environnement  expert,  desti- 
r.ees  a  definir  et  integrer  de  nouvelles  logiques  de  raisonnements,  a  verifier 
leur  coherence,  et  a  les  valider.  Cela  necessite  des  possibilites  de  saisie, 
de  visualisations,  et  de  controle  de  I'expertise. 

2.  Une  autre  facette  de  la  maquette  comprend  les  fonctions  d'environnement 
operationnel,  permettant  de  tester,  d'evaluer  et  de  valider  les  resultats  des 
raisonnement  experts,  en  terme  de  qualite  operationnelle  comme  en  terme 
de  temps  de  reaction.  Cette  partie  necessite  une  simulation  simplifiee  mais 
realiste  du  SNA  et  du  theatre  tactique. 

3.  On  assure  des  fonctions  d'environnement  informatique,  permettant  outre  le 
lancement  de  ('application,  I'archivage  des  sessions  en  vue  du 
depouillement  des  resultats,  le  monitoring  des  sessions  ainsi  que  la  gestion 
de  plusieurs  versions  des  expertises  codees  dans  la  maquette. 

4.  Enfin  on  dispose  de  fonctions  de  communication  permettant  de  dialoguer  et 
d'echanger  des  informations  entre  simulation  operationnelle,  simulation  du 
SNA,  Pilote  et  Copilote  Electronique.  Ces  logicieis  etant  realises  sur  machi¬ 
nes  differentes,  le  dialogue  etabli  consiste  a  un  echange  "en  ligne”  des 
informations  en  provenance  de  machines  heterogenes. 

Nous  ne  decrivons  ici  que  les  aspects  d'environnement  expert,  et  plus  jarticu- 

lierement  la  supervision  et  le  controle  des  entites  expertes. 


FONCTIONS  D'ENVIRONNEMENT  EXPERT 


Le  developpement  d'un  iogiciel  expert  comme  le  Copilote  Electronique  est 
caracterise  par  la  continuity  de  l'enrichissement  du  logiciel  entre  la  phase  d'ela- 
boration  des  iogiques  de  raisonnement  et  ia  phase  devaluation  et  de  validation 
de  ces  logiques.  L'ensemble  de  ces  fonctions  represente  la  partie  "embar- 
quabie"  du  concept  de  Copilote  Electronique,  a  I'exclusion  des  traitements  deja 
presents  dans  le  SNA.  Ces  fonctions  s'execulent  en  parallele  des  simulations 
de  maniere  a  pouvoir  evaluer  leur  applicability  au  probleme  temps  reel. 

Dans  le  cadre  de  la  maquette,  nous  avons  retenu  un  ensemble  coherent  de 
fonctionnalites  (parties  grisees),  respectant  I'architecture  fonctionnelle  globale 
presentee  ci-dessous.  Cet  ensemble  permet  de  presenter  des  aides  a  la  prise 
de  decision  tactique.  II  illustre  par  ailieurs  le  principe  d'echange  et  de  coope¬ 
rations  entre  les  entites  expertes  retenues.  L'ensemble  des  fonctionnalites  rete¬ 
nues  est  le  suivant  : 

•  Niveau  Reflexion  : 

«  Evaluation  de  ia  situation  tactique 

*  Evaluation  des  plans  de  mission 

•  Niveau  Decision  : 

«  Gestion  tactique 


PU.OTE  ENVIRONNEMENT  AVION 


figure  1.  Architecture  du  Copilote  Electronlque 

•  Gestion  des  informations  pertinentes 

Description  fonctionnelle 

Niveau  Reflexion 

Ce  niveau  regroupe  les  raisonnements  permettant  de  manipuler  I'ensemble 
des  donnees  dispombles  sur  I'environnement  reel  du  decideur.  II  correspond  a 
I'activite  d' information,  qui  consiste  a  traduire  sous  forme  symbolique  les  phe- 
nomenes  observes,  ainsi  qu'a  I'activite  de  prevision  qui  consiste  a  construire 
des  schemas  du  futur  probables.  On  elabore  ainsi  une  comprehension  de  I'en- 
vironnement  operationnel,  ainsi  que  des  predictions  devolution  de  la  mission. 
La  finalite  de  ces  raisonnements  est  d'elaborer  des  elements  de  decision  riches 
et  efficaces  a  I'usage  du  niveau  de  decision. 

1.  Evaluation  de  la  situation  tactlque 

L'evaluation  de  la  situation  tactique  revient  a  une  correlation  "intelli- 
gente"  continuellement  rafraichie  de  toutes  les  informations  disponibles  sur 
le  cadre  operationnel  (objectif,  dispositif  ennemi,  dispositif  ami).  Cela  sup¬ 
pose  : 

a.  la  synthese  des  informations  disponibles, 

b.  I'analyse  des  elements  de  la  situation  tactique, 

2.  Evaluation  des  plans  de  mission 

Cette  tache  maintient  en  permanence  un  ensemble  de  plans  de  mission 
envisaaeables  pour  la  poursuite  de  la  mission.  Pour  cela,  clle  modelise  les 
divers  plans  issus  de  la  preparation  de  mission  ou  d'une  replanification  en 
vol.  Elle  evalue  les  plans  de  mission  suivant  deux  criteres  qui  sont  le  risque 
d'une  part  et  I'elficacite  d'autre  part.  Le  risque  correspond  a  la  possibility 
de  ne  pas  mener  la  mission  a  son  terme.  L'efficacite  correspond  au  resultat 
attendu  de  I'attaque  pour  un  plan  de  mission  donne  en  fonction  de  I'objectif 
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fixe  en  preparation  de  mission  ou  en  cours  de  vol.  Les  fonctions  assurees 
sont  les  suivantes  : 

a.  Mise  a  lour  des  plans  de  mission 

b.  Mise  a  jour  des  m  rges 

c.  Mise  a  jour  des  risques 

d.  Information  du  reste  du  Copilote  Electronique 

Niveau  Decision 

Ce  niveau  regroupe  d'une  part  les  raisonnements  permettant  de  gerer  I'en- 
semble  des  informations  elaborees  par  le  niveau  de  reflexion  suivant  les  types 
de  decisions  a  prendre.  L'activite  d'elaboration  consiste  a  traduire  les  inten¬ 
tions  en  actes  envisageables  ou  propositions  d'actions,  l'activite  d' execution 
consiste  ensuite  a  interagir  avec  le  pilote  et  avec  le  systeme  de  maniere  a  pro¬ 
duce  des  resultats  efficaces,  et  agir  sur  le  monde  reel. 

On  genere  des  propositions  d'actions  adaptees  a  I'environnement  ope- 
rationnel  et  permettant  de  realiser  efficacement  la  mission.  La  finalite  de  ces 
raisonnements  est  de  fournir  des  informations  pertinentes  (propositions  d'ac¬ 
tions  ou  criteres  de  decision)  en  fonction  de  la  situation  et  des  besoins  du  deci- 
deur.  A  ce  niveau  sont  elaborees  et  choisies  les  solutions  aux  divers  problemes 
rencontres  pendant  le  deroulement  de  la  mission,  c'est  le  niveau  des  modules 
de  gestion. 

On  genere  des  propositions  d'actions  adaptees  a  I'environnement  ope- 
rationnel  et  permettant  de  realiser  efficacement  la  mission.  La  finalite  de  ces 
raisonnements  est  de  fournir  des  informations  pertinentes  (propositions  d'ac¬ 
tions  ou  criteres  de  decision)  en  fonction  de  la  situation  et  des  besoins  du  deci- 
deur. 

•  Les  modules  gestion  avion,  gestion  tactique  et  gestion  mission  traitent  en 
entree  les  informations  symboliques  et  fournissent  en  sortie  d'autres  infor¬ 
mations  symboliques  de  type  proposition  d'action  ou  explication  de  deci¬ 
sion. 

•  Le  module  de  gestion  des  informations  pertinentes  traite  en  entree  les  infor¬ 
mations  symboliques  et  fournit  en  sortie  des  informations  pertinentes  de 
decision  ou  de  synthese,  ainsi  que  des  informations  symboliques  sur  le  dia¬ 
logue  en  cours.  Ces  informations  sont  dites  pertinentes  car  elles  ne  sont 
dirigees  vers  le  pilote  via  I'interface  qu'a  bon  escient,  c'est  a  dire  au  bon 
moment  et  par  le  bon  support. 

A  ce  niveau  sont  elaborees  et  choisies  ies  solutions  aux  divers  problemes  ren¬ 
contres  pendant  le  deroulement  de  la  mission,  c'est  ie  niveau  des  modules  de 
gestion. 

1.  Gegtion  tactique 

Ce  module  determine  les  actions  adaptees  aux  tactiques  defensives  t* 
offensives  face  k  des  menaces  ou  des  objectifs  en  fonction  du  cadre  ope- 
rationnel  rencontre  au  cours  de  la  mission  (mise  en  oeuvre  d'une  arme,  pro¬ 
cedure  d'attaque,  affectation  de  cible,  manoeuvre  en  duel  aerien,  evasive 
face  a  un  missile,  brouillage  et  leurrage...).  Le  traitement  a  effectuer  et  I'am- 
pleur  des  logiques  mises  en  oeuvre  dependent  de  la  pression  temporelle. 

a.  Determination  de  la  pression  temporelle 
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b.  Selection  de  la  menace  a  tralter 

c.  Cholx  de  parade 

d.  Determination  des  parades  compatibles 

e.  Determination  de  I'efficacite  des  parades  selectionnees 

f.  Determination  de  parades  hors  marge  si  besoin 

g.  Envoi  des  proposition  de  parades 

2.  Gestton  des  informations  perdnentes 

il  s'agit  de  gerer  les  echanges  Copilote  -  SNA,  et  en  particulier,  les 
echanges  Copilote  -  Pilote,  via  I'lHM.  Ce  module  determine  les  actions  du 
Copilote  Electronique  et  gere  !e  dialogue  et  les  passages  de  commandes  qui 
en  decoulent.  Pour  cela,  il  gere  les  informations  symboliques  pour  fournir 
au  pilote  ies  informations  consid6rees  comme  les  plus  importantes  a  tout 
moment.  II  gere  les  eventuels  conflits  de  decisions  en  prenant  en  compte 
les  facteurs  humains  dans  I'assistance  au  pilote.  II  definit  le  type  de  dialo¬ 
gue  le  plus  ad6quat  en  fonction  de  I'activite  du  pilote,  de  i'urgence  de  la 
situation  el  de  I'etat  ce  I'lnterface  Homme-Machine.  Enfin,  il  permet  de 
varier  le  niveau  d'alde  a  apporter  au  pilote,  en  definissant  dynamiquement 
les  actions  possibles  en  automatique. 

a.  Filtrage  des  informations  symboliques, 

b.  Dialogue  pilote  /  Copilote  Electronique. 

Supervision  des  traitements 

Niveau  Reflexion 

Les  modules  du  niveau  de  reflexion  doivent  apporter  une  vision  globale  et 
coherente  du  monde,  et  maintenir  cette  vision  incrementalement.  Ceci  amene 
naturellement  a  penser  a  une  architecture  a  base  de  blackboard,  permettant  la 
cooperation  de  diverses  entites  expertes.  Les  problemes  de  definition  d'une 
architecture  de  blackboard  pour  le  niveau  de  reflexion  du  Copilote  Electronique 
sont  essentiellement  les  suivants  : 

•  problemes  lies  a  une  architecture  parallele  et  non  pas  sequentielle,  comme 
la  majorite  des  systemes  a  base  de  blackboard  actuels, 

•  problemes  lies  a  la  cooperation  entre  une  architecture  a  base  de  blackboard 
pour  le  niveau  de  reflexion,  et  une  architecture  a  base  d'interruptions  pour 
le  niveau  de  decision  (voir  plus  loin), 

roblemes  lies  a  I'integration  dans  un  univers  evolutif  et  a  des  raison- 
ements  non  monotones. 

/Oints  d'etude  font  I'objet  d'un  contrat  DRET(”  avec  I'Onera12’  (89-34347).  A 
I'heure  actuelle,  une  solution  simple  de  controle  par  niveau  de  priorite  a  ete 
retenue.  Cette  solution  sera  bien  entendu  amelioree  en  fonction  des  resultats  de 
I'etude  proposee. 

Niveau  Decision 


A  la  difference  des  modules  du  niveau  de  reflexion,  les  modules  du  niveau 
de  decision  sont  beaucoup  plus  independants  les  uns  des  autres,  ils  n'ont  pas 
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besoin  de  communiquer  beaucoup  entre  eux  pour  mener  a  bien  leur  tache  res¬ 
pective  (“loosely  coupled”),  ce  qui  amene  a  concevoir  une  architecture  de 
controle  differente. 

Pour  chaque  module  de  decision,  il  est  necessaire  d'evaluer  le  temps  de 
decision  disponible,  done  de  determiner  quelle  est  la  pression  temporelle  pour 
une  situation  donnee.  En  effet  II  taut  avoir  un  ordre  de  grandeur  du  temps  dont 
on  dispose  pour  elaborer  une  decision.  De  plus,  il  faut  pouvoir  interrompre  un 
raisonnement  lorsque  le  temps  impart!  est  ecoule.  Actuellement,  on  distingue 
trois  valeurs  pour  la  pression  temporelle  : 

L'urgence  Elle  caracterise  toute  situation  necessitant  un  raisonnement  rapide 
conduisant  a  des  actions  reflexes  consecutives  a  des  risques  majeurs 
ayant  fait  I'objet  d'une  analyse  prealable.  Cette  analyse  a  pu  avoir  lieu 
anterieurement  au  cours  de  I'execution  de  la  mission  (prevision)  ou  etre 
memorisee  (conditionnement)  des  la  conception. 

La  pression  temporelle  moyenne  Elle  intervient  dans  les  cas  ou,  bien  que  la 
situation  soit  assez  critique,  il  est  possible  d'entreprendre  des  raison- 
nenients  plus  fouilles  conduisant  a  des  actions  a  court  terme.  Apres  avoir 
analyse  les  differentes  possibilites,  on  procede  rapidement  a  une  selection 
d'un  des  comportements  possibles. 

La  pression  temporelle  faible  conduit  a  une  planification  d'ordre  strategique  sur 
la  base  d'informations  plus  raffinees  et  conduisant  a  des  actions  com¬ 
plexes.  On  peut  s'y  livrer  a  une  analyse  approfondie  de  la  situation  et  en 
deduire  des  mises  a  jour  sur  le  plan  de  mission  tenant  compte  des  diver- 
ses  repercussions  a  moyen  et  long  terme.  C'est  notamment  en  pression 
temporelle  faible  qu'il  est  possible  de  preparer  les  comportements  reflexes 
a  adopter  dans  les  situations  d'urgence  en  prenant  en  consideration  suffi- 
samment  de  parametres. 

En  fonction  des  evenements  ou  des  modifications  induisant  des  repercus- 
ions  importantes  (diminution  d'efficacite  ou  risque  important),  on  determine  le 
module  induisant  la  pression  temporelle  la  plus  importante.  Le  controle  lui  est 
alors  transmis  pour  effectuer  les  raisonnements  d'aide  a  la  decision.  La  notion 
de  pression  temporelle  serf  ici  une  premiere  fois  pour  la  resolution  de  conflits 
externe,  c'est  a  dire  pour  choisir  le  module  devant  apporter  I'aide  a  la  decision 
la  plus  urgente. 

Le  terme  "imprevu"  apparaissant  dans  ce  document  doit  etre  pris  au  sens 
large  comme  definissant  tout  evenement  imprevu,  devant  faire  I'objet  d'une  pri¬ 
se  en  charge  par  le  Copilote  dans  I'un  de  ses  modules  de  decision  (Gestion 
Tactique,  Gestion  Mission,  Gestion  Avion  et  Gestion  des  Informations  Perti- 
nentes).  La  prise  en  charge  de  tels  evenements  doit  aboutir  en  fin  de  traitement 
a  une  proposition  de  decision  a  transmettre  au  pilote. 

•  Sequencement  et  priorites 

La  decomposition  fopctionnelle  du  niveau  de  decision  fait  apparaitre 
des  taches  d'aide  a  la  decision  pour  des  domaines  d'expertise  independants 
les  uns  des  autres.  Par  ailleurs,  chaque  tache  elementaire  d'un  module 
d'aide  a  la  decision  fait  appel  a  des  traitements  informatiques  independants. 
II  s'agit  ainsi  d'activer  des  traitements  pour  toutes  les  taches  connues  a  un 
instant  donne.  Les  problemes  devant  etre  pris  en  compte  sont  les  suivants  : 

«  Determination  de  la  pression  temporelle  liee  a  chaque  evenement  impre¬ 
vu. 
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Determination  de  I'imprevu  a  traiter. 
Lancement  du  ou  des  traitements  a  appliquer. 


Cependant  le  temps  necessaire  a  ('elaboration  d'une  proposition  d'ac- 
tion  est  essentiellement  variable.  II  est  necessaire  de  tenir  compte  de  la 
pression  temporelle,  done  du  temps  imparti,  pour  decider  de  lancer  ou  non 
des  traitements  elabores.  Lorsque  le  temps  imparti  est  ecoule  (et  qu'il  taut 
presenter  a  tout  prix  une  proposition  d'action),  le  module  doit  achever  ses 
traitements  en  fournissant  un  resultat,  fut-il  de  moins  bonne  qualite  :  une 
parade  tres  efficace  mais  proposee  trop  tard  n'est  en  fait  d'aucune  utilite. 

Pour  permettre  un  comportement  adaptatif  et  opportuniste,  dont  les  temps 
de  reponse  soient  maitrises,  ii  faut  pouvoir  disposer  des  fonctionnalites  sui- 
vantes  : 

■  Interruptions  des  raisonnements  et  des  traitements  selon  les  chan- 
gements  d'environnement  et  de  pression  temporelle  associee. 

■  Interruptions  des  raisonnements  et  des  traitements  en  cas  de  depas- 
sement  du  delai  imparti. 

■  Masquage  des  interruptions  dans  certaines  phases  critiques. 

•  Cas  nominal 

Reprenons  et  analysons  chacun  de  ces  points  dans  le  cas  d'un  fonction- 
nement  nominal,  e'est  a  dire  sans  perturbation  des  traitements  : 

1.  Determination  de  la  pression  temporelle  de  chaque  evenement  imprevu 

Ce  niveau  de  pression  temporelle,  prenant  trois  valeurs  (faible, 
moyen,  fort)  est  determine  par  le  niveau  de  risque  induit  par  I'evene- 
ment  a  traiter,  la  position  de  ('avion,  I'urgence  de  la  decision. 

Cette  tache  peut  etre  declenchee  a  loute  arrivee  de  nouvel  imprevu 
susceptible  d'avoir  des  repercussions  importantes,  et  d'augmenter  la 
pression  temporelle  ;  elle  peut  etre  consideree  comme  une  tache  privile- 
giee,  pouvant  se  declencher  a  tout  moment  et  pouvant  modifier  les  trai¬ 
tements.  A  la  fin  de  cette  tache,  la  main  est  rendue  au  traitement 
precedemment  en  cours. 

2.  Determination  de  1'evenement  imprevu  a  traiter 

II  s'agit  d'ordonner  I'ensemble  des  imprevus  a  traiter,  en  les 
indexant  par  leurs  niveau  de  pression  temporelle,  afin  de  choisir  I'impre- 
vu  a  traiter,  qui  devient  “I'imprevu  courant”.  II  faut  eventuellement  deci¬ 
der  de  suspendre  le  traitement  de  I'imprevu  en  cours  (imprevu  1)  pour 
traiter  un  imprevu  plus  prioritaire  (imprevu  2). 

«  Dans  ce  cas  il  faut  donner  la  main  a  ce  nouvel  imprevu  (2).  Lorsque 
cet  imprevu  aura  ete  traite,  la  tache  “Choix  de  i'imprevu”  sera  reacti- 
vee  pour  decider  de  I'imprevu  a  traiter. 

«  Dans  le  cas  contraire  on  revient  au  point  du  traitement  ou  I'on  s'etait 
arrete  pour  I'imprevu  1. 

3.  Traitement  de  I'imprevu 

Au  cours  de  cette  tache  sont  lances  successivement  des  traitements 
qui  ont  pour  but  de  proposer  une  decision.  Ces  differents  traitements 
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correspondent  a  des  etapes  fonctionnelles  du  raisonnement  ;  ils  sont 
implantes  de  fapon  telle  qu'ils  affinent  progressivement  la  proposition  de 
decision,  De  cette  fagon  ils  peuvent  etre  inforrompus  a  tout  moment  en 
cas  de  temps  imparti  ecoule,  et  fournir  un  resultat. 


Cti8  d'exceptlons 

II  est  necessaire  de  definir  un  systeme  d'interruptions  de  raison- 
nements  qui  doivent  se  declencher  dans  les  cas  suivants  : 

•  lorsque  la  pression  temporelle  induite  par  I'evenement  imprevu  augmen- 
te  en  dynamique. 


C'est  le  cas  par  exemple  lorsqu'une  menace  tire  son  missile  et  done 
provoque  i„ne  augmentation  du  risque,  ou  lorsque  le  segment  courant 
devient  celui  de  la  menace,  induisant  une  pression  temporelle  forte. 
Dans  ce  cas  II  peut  etre  necessaire  d'abreger  les  traitements  en  fournis- 
sant  une  proposition  de  parade  sans  delai. 


lorsque  I'imprevu  a  trailer  change. 


(Un  autre  imprevu  presente  une 


pression  temporelle  superieure  a  I'imprevu  courant). 


C'est  le  cas  iorsqu'un  evenement  grave  ou  une  menace  survient 
pendant  un  traitement  de  moindre  importance.  La  pression  temporelle 
de  I'imprevu  dont  le  traitement  est  en  cours  n'est  plus  la  pression  tem¬ 
porelle  la  plus  eievee,  II  taut  done  stopper  le  raisonnement  en  cours 
(quitte  a  le  reprendre  plus  tard),  pour  elaborer  une  proposition  de  para¬ 
de  sur  le  nouvel  imprevu. 


*  lorsque  le  temps  imparti  est  ecoule. 


Le  systeme  doit  imperativement  proposer  un  resultat  dans  un  delai 
determine  a  I'avance.  A  I'echeance  fixee  le  traitement  doit  s'interrom- 
pre  (tout  traitement  ulterieur  ne  serait  d'aucune  utilite  et  done  serait  du 
temps  machine  perdu). 


Toutefois,  il  est  necessaire  de  prevoir  la  possibility  de  masquer  ses 
interruptions  du  raisonnement.  Sans  cela  le  module  risque  de  s'interrompre 
continuellement  dans  certains  cas,  et  done  de  ne  jamais  pouvoir  fournir  le 
moindre  resultat. 


«  Dans  les  premier  et  troisieme  cas  d'interruptions,  le  raisonnement  est  en 
fait  stoppe  ou  abrege  pour  fournir  une  proposition  de  rysultat.  Le  modu¬ 
le  rend  ensuite  la  main  pour  d'autres  traitements.  L'interruption  ne  pose 
pas  de  problemes  de  masquage. 

•  Dans  le  second  cas,  le  masquage  d'interruptions  repete  est  garanti  par 
le  nombre  fini  de  valeurs  de  la  pression  temporelle.  Si  le  module  etait 
en  train  de  traiter  un  imprevu  sous  pression  tempo^eHe  faible,  il  risque 
au  pire  d'interrompre  ses  traitements  deux  fois  :  une  premiere  fois  pour 
un  imprevu  sous  pression  temporelle  moyenne  e»  une  seconde  fois  pour 
un  imprevu  sous  pression  temporelle  forte. 

Ensuite  le  module  masque  toute  nouveile  interruption  :  I'idee  sous- 
jacente  est  qu'en  cas  d'existence  de  plusieurs  imprevus  induisant  toutes 
une  forte  pression  temporelle,  il  vaut  mieux  continuer  le  raisonnement 
sur  celui  pour  qui  on  est  le  plus  avance,  done  sur  I'imprevu  en  cours  de 
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traitement,  car  c'est  pour  lui  qu'on  a  le  plus  de  chances  de  trouver  rapi- 
dement  une  proposition  d'action  acceptable.  De  plus,  cette  notion  de 
poursuite  du  raisonnement  est  conforme  a  I'idee  d'une  ligne  de  raison- 
nement  unique,  pour  le  pilote  comme  pour  le  copilote  eiectronique,  pen¬ 
dant  les  phases  critiques. 


PERSPECTIVES 


•  Un  systems  “Human-like” 

Les  systemes  d'aides  se  differencient  des  automates  de  par  la  presence 
de  I'operateur,  veritable  acteur  temps  qui  doit  faire  face  a  la  complexite  des 
systemes  et  agit  directement  sur  la  dimension  temporelle  du  processus.  II 
est  done  important  de  tenir  compte  de  cette  dimension  dans  I'interaction  du 
pilote  et  du  "Copilote  Eiectronique"  car  il  semble  qu'une  grande  partie  des 
erreurs  humaines  soit  due  a  une  mauvaise  apprehension  du  temps. 

Nous  travaillons  dans  ce  sens  avec  le  CERMA(3)  pour  etudier  un  modele 
de  comportement  du  pilote,  reprenant  des  heuristiques  humaines  de  gestion 
du  temps. 

•  Specifications  temps  reel 

Les  mecanismes  d'interruptions  sont  inspires  de  ceux  utilises  dans  les 
systemes  temps  reel.  Cependant  I'analogie  s'arrete  la,  nous  n'envisagerons 
ici  que  les  contraintes  de  synchronisation  sur  une  horloge  (qui  dans  notre 
cas  n'est  pas  temps  reel).  La  conception  de  modules  experts  capables  d'in¬ 
terruptions  et  de  synchronisation  sur  une  horloge  donnee  permettra  pai  la 
suite  d'envisager  des  capacites  temps  reel  au  sens  habituel  du  terme  (c'est 
a  dire  dont  les  temps  de  reponse  sont  non  seulement  maitrises,  mais  aussi 
et  surtout  tres  courts). 

La  description  de  tels  mecanismes  n'est  pas  une  specification  logicielle. 

On  n'a  pas  decrit  comment  devaient  etre  codes  les  mecanismes  presentes, 
mais  seulement  les  proprietes  que  devaient  verifier  les  mecanismes  d'aide  a 
la  decision  du  Copilote  Eiectronique.  Le  fonctionnement  du  controle  des 
activites  d'aides  a  la  decision  est  guide  par  la  notion  de  pre.;sion  tempo- 
reile.  II  s'appuie  sur  des  mecanismes  d'intenuptions.  Dans  ce  cadre,  la  ges¬ 
tion  des  taches  en  concurrence  informatique  peut  s'apparenter  a  une  gestion 
de  systeme  d'exploitation. 

•  Optimisation  d'architecture 

Enfin,  nous  menons  avec  I'Onera®  une  etude  visant  a  ameliorer  I'archi- 
tecture  multi-expert  actuellement  retenue  pour  le  Copilote  Eiectronique,  au 
moycn  d'un  simulateur.  II  permettra  de  choisir  les  granularites  des  trai- 
tements,  d'analyser  les  flux  et  les  interactions  entre  les  entites  expertes  et 
de  maitriser  la  complexite  de  la  supervision 

i 

* 


{3>  Centre  d'Etudes  et  He  Recherches  de  M6decine  Aerospatiale 
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1.  INTRODUCTION 

One  of  the  main  problems  arising  in  the  field  of  missile  guidance  is  the 
automatic  search  and  detection  of  targets,  in  order  to  gather  the  necessary 
information  for  the  correct  homing  of  the  missile. 

This  problem  is  commonly  approached  by  mounting  a  seeker  (usually  an  IR  seeker) 
with  an  image  and  data  processor  on  board  of  the  missile. 

The  ground  scene,  as  recorded  by  the  seeker,  will  present  isolated  targets  and 
target  formations,  together  with  a  high  level  of  clutter  which  produces  a  number  of 
false  alarms.  Thus,  it  is  necessary  to  provide  the  image  and  data  processor  of  the 
missile  with  algorithms  which  can  automatically  eliminate  the  clutter  and  the  false 
alarms  and  detect  the  true  targets. 

Furthermore,  in  order  to  have  an  effective  shot,  only  one  formation  among  the 
various  in  the  scene  must  be  cued  to  the  navigation  and  the  weapon  systems  of  the 
missile,  the  choice  of  it  depending  on  the  shape,  the  disposition  and  the  distance 
between  targets  of  the  formation  itself. 

This  paper  presents  an  algorithm  developed  to  evaluate  the  optimal  point  for 
releasing  the  ammunition.  The  algorithm  is  further  illustrated  by  applying  it  to  a 
practical  case  and  showing  the  results  of  this  simulation. 

The  paper  is  based  on  the  work  carried  out  by  Italy  within  the  NATO  Research 
Study  Group  on  Image  Processing,  RSG.9  -  Project  4. 


2.  BACKGROUND  AND  GENERAL  STRUCTURE  OF  THE  ALGORITHM 

The  starting  point  for  the  algorithm  is  a  file  of  data  obtained  from  the 
preprocessing  of  a  sequence  of  images. 


The  length  of  such  a  sequence  depends  on  different  aspects,  such  as  the  dynamics 
of  the  recorded  scene  (the  higher  the  dynamics,  the  shorter  the  sequence),  the  action 
range  of  the  missile  and  its  flight  characteristics  (altitude,  speed),  the  computing 
speed  of  the  processor  on  board  of  the  missile. 

The  preprocessing  of  the  images,  consisting  of  segmentation,  filtering, 
thresholding,  labelling,  background  registration  and  prediction  of  the  position  of 
those  targets  which  are  temporarily  obscured  (for  example  by  smoke  or  vegetation)  in 
some  frames  of  the  sequence,  will  supply  the  algorithm  with  the  labelled  coordinates 
of  the  detected  hot-spots. 

The  analysis  will  be  performed  on  this  set  of  data,  schematically  shown  in 
Figure  1 . 

The  mathematical  model  used  is  based  on  the  consideration  that  targets  belonging 
to  the  same  formation  show  some  similar  features,  such  as  speed  and  mutual  distance. 
For  this  reason  the  algorithm  establishes,  also  by  means  of  a  comparison  with 
predefined  data  and  characteristics,  which  is  the  range  of  values  for  these  figures 
that  characterizes  each  group,  thus  being  able  to  reject  false  alarms,  to  identify 
isolated  targets  and  to  group  the  remaining  targets  in  formations. 

Another  comparison  between  predefined  data  of  the  missile  weapon  system  and  some 
evaluated  figures  will  finally  lead  to  the  location  of  the  optimal  formation  and, 
within  it,  of  the  optimal  point  for  munitioning. 


FIGURE  2  -  ALGORITHM  GENERAL  STRUCTURE 
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Figure  2  shows  the  general  structure  of  the  algorithm.  The  calculation  performed 
and  the  criteria  used  to  implement  the  grouping  process  are  discussed  in  details  in 
the  subsequent  sections. 


3.  INTER-FRAMES  ANALYSIS:  GROUPING  BY  SPEED 

During  the  first  step  of  the  algorithm,  the  hot-spots  are  grouped  according  to 
the  similarity  of  their  speed. 

For  each  point  i  of  the  input  file,  the  algorithm  evaluates  the  relevant  total 
displacement  s(i)  (i.e.  the  distance,  in  pixels,  covered  by  the  point  throughout  the 
sequence)  as  the  sum  of  the  frame  by  frame  displacements. 

The  partial  displacements  are  the  instantaneous  speed  values  (in  pixels/frame)  of 
the  points:  each  of  these  figures  is  compared  with  a  predefined  maximum  speed  value, 
available  to  the  data  processor  and  evaluated  as  a  function  of  the  IR  seeker  altitude 
over  the  ground,  of  its  Instantaneous  Field  of  View,  of  the  frame  time  and  of  the 
foreseen  maximum  target  speed  on  the  ground  (km/h).- 

Those  hot-spots  having  a  speed  value  greater  than  the  predefined  one  are  rejected 
as  false  alarms. 

The  remaining  points  are  then  subdivided  into  groups  by  means  of  the  method 
described  in  the  following. 

The  mean  value,  /a(s).  and  the  standard  deviation,  <T(s),  of  the  set  of  total 
displacements  are  evaluated: 


N 

^i(s)  =  1/N  £1  s ( i )  , 


1  /  N  2 

<J(s)  =  U  1/N  [s(i)  -  ju(s)]  , 

where : 

i  =  label  of  the  hot-spot, 

N  =  total  number  of  hot-spots. 

If  the  standard  deviation  is  small: 

d(s)  <  d*(s), 

where  o*(s)  is  a  predefined  threshold  value,  then  the  hot-spots  show  similar 
displacement  values  in  the  sequence,  i.e.  they  have  similar  speed,  and  they  are 
considered  to  belong  tc  the  same  group. 

If,  on  the  contrary,  the  standard  deviation  value  is  greater  than  the  threshold 
level,  the  points  are  subdivided  into  groups,  according  to  the  positive  or  negative 
sign  of  the  difference: 

[s(i)  -  ju(s)  ]  . 

In  this  second  case,  the  process  is  repeated  once  more  on  each  set  obtained,  to 
confirm  the  validity  of  the  grouping,  or  to  perform  a  further  subdivision. 

Isolated  targets  obtained,  in  case,  after  the  subdivision  are  rejected.  A 
rejection  can  be  performed  also  on  groups  of  two  targets. 

Simulation  trials  carried  out  to  test  the  algorithm  have  shown  that  a  threshold 
value  that  can  be  used  for  the  standard  deviation  is: 

d*(s)  w  0.2  •  ji(s). 


4.  FRAME  BY  FRAME  ANALYSIS:  GROUPING  BY  DISTANCE 

In  the  second  part  of  the  algorithm  the  analysis  proceeds  frame  by  frame  and 
separately  on  each  group  identified,  in  case,  in  the  previous  step. 

In  each  frame  of  the  sequence  the  algorithm  evaluates  the  mutual  distances  of 
targets  and,  following  the  assumption  that  targets  belonging  to  the  same  formation 
have  homogeneous  distances,  subdivides  the  targets  accordingly. 

To  accomplish  this  task,  the  distances  between  each  target  and  all  the  others  are 
evaluated  and,  for  every  target  j,  only  the  shortest,  d(j),  is  retained.  Proceeding 
this  way,  all  the  M  targets  are  connected  in  a  chain  of  (M-l)  sides,  in  which  every 
point  is  linked  to  its  neighbour.  Within  this  chain,  targets  can  be  classifte.  as: 

-  End  Point  If  the  point  is  linked  only  to  one  other  point; 

-  Intermediate  Point  if  the  point  as  linked  to  two  other  points- 

-  Branch  Point  if  the  point  is  linked  to  more  than  two  o  r  points. 

Figure  3  gives  an  example  of  a  chain  of  targets:  End  Points  aie  marked  with  EP 
and  Branch  Points  with  BP,  the  othe’-  points  are  Intermediate  Points. 

The  analysis  proceeds  with  the  evaluation  of  the  mean  value,  p(d),  and  the 

standard  deviation,  d(d),  of  the  set  of  minimum  distances: 
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M-1 

ji(d)  =  1/ ( M— 1 )  ^  d(j)  , 

■>  M-1  7 

d(d)  =  Ul/fM-l)  E  [d(  j)  -  ji(d)]  . 


FIGURE  3  -  CHAIN  OF  TARGETS  (BP=Branch  Point,  EP=End  Point) 


If  the  standard  deviation  is  small: 

<J(d)  <  d*(d), 

where  d*(d)  is  a  predefined  threshold  value,  then  the  targets  show  similar  distances, 
i.e.  they  belong  to  the  same  formation,  and  the  analysis  proceeds  on  the  next  frame. 

If,  on  the  contrary,  the  standard  deviation  value  is  greater  than  the  threshold 
level,  then  a  counter  K  nlevant  to  the  group  under  examination  is  initialized  and 
the  following  positive  distances  are  evaluated: 

(1)  |  d(j)  -  ji(d)  |  j  =  1,...,M-1. 

If,  for  a  distance  d(j),  the  value  of  the  difference  in  (1)  is  less  than 
( 1+Kc)  •  d(d)  ,  whore  c  >  0  is  a  predefined  value,  then  the  corresponding  link  is 
retained.  On  the  other  hand,  differences  greater  than  (l+Kc)-<J(d)  are  analyzed  and 
a  cut  is  produced  in  the  chain  on  the  link  relevant  to  the  largest  difference. 

The  whole  process  is  repeated  iteratively  on  each  group  obtained,  incrementing  of 
one  unit  the  relevant  counter  at  each  step,  until  the  current  value  of  did)  is 
smaller  than  d*(d). 

when  a  cut  produces  an  isolated  target,  i.e.  the  cut  is  made  on  the  link  of  an 
End  Point,  this  target  is  rejected;  once  again  it  can  be  decided  to  reject  also 
groups  made  up  of  only  two  targets. 

The  groups  of  targets  obtained  after  the  execution  of  the  described  process 
represent  the  formations  of  the  current  frame.  The  analysis  proceeds  analogously  on 
each  frame  of  the  sequence. 

Simulation  trials  carried  out  to  test  the  algorithm  have  led  to  the  following 
settings  of  the  predefined  constants: 

d*(d)  *  0.2  •  /a(d)  , 

c  0.2  . 

The  counter  K  has  been  introduced  in  the  calculation  to  reflect  the  decrease  of 
probability  of  further  sub-divisions  in  the  current  group,  as  the  number  of  performed 
iterations  increases. 


5.  ASSIGNMENTS  OF  THE  TARGETS  TO  THE  FORMATIONS 

As  it  has  been  shown  in  the  previous  section,  the  execution  of  the  algorithm,  to 
this  extent,  leads  to  the  determination  of  the  formations  present  in  each  frame. 

However,  due  to  the  movement  of  the  targets,  different  sub-divisions  may  be 
operated  by  the  processing  in  the  various  frames.  For  this  reason,  a  track  is  kept  of 
all  the  configurations  obtained,  so  tnat,  at  the  end  of  the  sequence,  the  algorithm 
is  able  to  decide  to  which  formation  esrh  target  must  be  assigned  to. 


This  is  accomplished  by  means  of  the  compilation  of  a  Frequency  Table  in  which 
the  algorithm  records  the  number  of  presences  of  each  target  in  the  formations.  Each 
presence  is  also  weighted  with  a  factor  F  that  increases  with  the  position  of  the 
current  frame  in  the  sequence:  the  closer  the  frame  to  the  end  of  the  sequence,  the 
higher  the  factor  F. 

At  the  end  of  the  sequence,  those  targets  having  the  sum  of  their  weighted 
presences  lower  than  a  certain  predefined  value  P  will  be  rejected,  and  each  of  the 
remaining  targets  will  be  assigned  to  the  formation  in  which  the  sum  of  its  weighted 
presences  is  higher. 

In  the  simulations  performed,  the  following  values  for  P  and  for  the  weighting 
factor  have  been  used: 

P  =  0.3  L  , 

F  =  1  +  1/L  , 

where: 

1  =  current  frame  number, 

L  =  total  number  of  frames. 


6.  EVALUATION  OF  THE  OPTIMAL  HOMING  POINT 

The  last  part  of  the  algorithm  is  devoted  to  the  individualization  of  the  most 
suitable  formation  to  be  shot  and,  within  it,  of  the  optimal  point  of  homing. 

This  is  accomplished  by  comparing  the  predefined  data  of  the  missile  weapon 
system  with  the  corresponding  characteristics  of  each  formation,  and  associating  a 
"score"  to  every  successful  comparison. 

One  characteristic  to  compare  is  the  distance  of  the  targets:  the  predefined 
datum  is,  in  this  case,  a  range  of  values  in  which  /i(d)  should  fall,  evaluated  (in 
pixels)  as  a  function  of  the  in  seeker  altitude  over  the  ground,  of  its  Instantaneous 
Field  of  View  and  of  the  distance  (in  meters)  of  the  points  shot  by  the  munitions. 
The  distance  between  End  Points  can  be  considered  too:  this,  together  with  the 
eventual  presence  of  Branch  Points,  gives  an  idea  on  how  much  the  formation  is 
stretched  out. 

A  check  will  be  done  also  on  the  number  of  targets  belonging  to  the  formations, 
determining  the  optimal  one  with  respect  to  this  feature. 

Finally,  the  shape  of  the  formations  will  be  recognized  by  evaluating  the  angles 
formed  by  the  targets,  taken  three  by  three.  If,  for  example,  a  wedge  formation  is 
preferred,  a  higher  score  will  be  associated  to  the  group  of  targets  that  can  be 
linked  in  such  a  way  to  present  all  the  angle  values  close  to  180°,  except  for  one 
angle,  corresponding  to  an  intermediate  target. 

Once  the  optimum  formation  has  been  selected,  the  individualization  of  the  homing 
point  (the  barycenter,  the  center  of  the  smallest  circumscribed  circle,  the  medium, 
etc.)  can  be  readily  made  with  simple  calculations  on  the  coordinates  of  the  targets. 
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7.  A  PRACTICAL  CASE:  EXPERIMENTAL  RESULTS 

In  this  section  the  algorithm  previously  described  is  illustrated  by  applying  it 
to  a  sequence  of  preprocessed  images  of  256  by  256  pixels. 

The  algorithm  has  been  implemented  in  FORTRAN  on  a  DIGITAL  VAX  780  computer. 

For  this  example,  a  quite  simple  sequence  has  been  chosen,  in  order  to  highlight 
the  mathematical  computation  performed  by  the  algorithm. 

Moreover,  in  order  not  to  be  tied  to  a  particular  situation  or  scenario,  the 
comparisons  with  predefined  values  of  the  IR  seeker  and  of  the  weapon  system  of  the 
missile,  have  not  been  accomplished. 

Figure  4  and  Figure  5  show  the  disposition  of  the  hot-spots  in  the  first  and  the 
last  image  of  the  sequence,  respectively. 

Thirteen  hot-spots  are  present  in  the  sequence,  numbered  with  the  labels  set  by 
the  preprocessing. 


FIGURE  4  -  LABELLED  HOT-SPOTS  IN  THE  FIRST  IMAGE  OF  THE  SEQUENCE 


FIGURE  5  -  LABELLED  HOT-SPOTS  IN  THE  LAST  IMAGE  OF  THE  SEQUENCE 


The  evaluation  of  the  total  displacements  of  the  points  lead  to  the  values  shown 
in  Table  1,  where  SUM(i)  is  the  name  used  by  the  computer  program  to  indicate  the 
distance,  in  pixels,  covered  by  the  i-th  point  throughout  tne  sequence  (i.e.  the  s(i) 
of  section  3.). 


SUM( 

1>  = 

16.62k 

SUM( 

2)  = 

61 .877 

SUM( 

3)  = 

20.105 

SUM< 

4)  = 

22.190 

SUM( 

5)  = 

15.318 

SUM( 

6)  = 

19.028 

SUM( 

7)  = 

19.385 

SUM( 

8>  = 

13.207 

SUMC 

9>  = 

19.868 

SUM<10)= 

18.000 

SUM(11)= 

60.908 

SUM( 12) = 

25.042 

SIJM(13)=  23.023 

TABLE  1  -  TOTAL  DISPLACEMENTS  OF  THE  HOT-SPOTS 


m 
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The  mean  value  and  the  standard  deviation  for  the  set  of  figures  in  Table  1  are: 
/Ms)  =  25.74  , 

(J(s)  =  15.50  , 

and  the  threshold  value  for  cMs)  is: 


<5*  ( s>  =  0.2  ■■  /M  s )  =  5.15  . 

Being  the  standard  deviation  greater  than  <5*(s),  the  set  of  points  must  be 
subdivided  according  to  the  sign  of  the  differences:  [SUM(i)  -  ju(s))  . 

Two  groups  are  obtained:  the  first  one  composed  by  eleven  points  (number  1,  3,  4, 
5,  6,  7,  8,  9,  10,  12  and  13)  and  the  second  one  composed  by  the  remaining  two 
targets. 

The  values  of  /Ms),  d(s)  and  d*(s)  for  the  first  group  are: 

/Ms)  =  19.25  , 

<J(s)  =  3.28  , 

<J*(s)  =  3.85  , 

and  for  the  second  group: 

/Ms)  =  61.39  , 

<J(s)  =  0.48  , 

<5*(s)  =  12.28  . 


In  both  cases,  no  further  subdivision  is  necessary;  the  second  group  (hot-spots 
number  2  and  11)  is  rejected  and  the  remaining  points,  subject  of  the  subsequent 
analysis,  are  re-labelled. 

The  algorithm  considers  now  the  first  frame:  it  evaluates  the  distances  from 
point  number  1  to  all  the  other  points  and  retains  the  shortest,  which  lead  to  point 
number  2,  as  shown  in  Figure  6. 


FIGURE  6  -  MINIMUM  DISTANCE  EVALUATION 


The  composition  of  the  chain  proceeds  following  three  rules: 

-  The  algorithm  is  not  allowed  to  close  loops,  nor  to  isolate  couples  of  points:  if 
point  2  is  linked  to  point  4,  point  4  must  be  either  linked  to  another  point  not  jet 
in  the  chain  or  be  an  End  Point,  but  cannot  be  linked  back  to  point  2. 

-  For  every  new  point  to  be  added  in  the  chain,  a  check  is  done  whether  this  point 
can  be  linked  with  a  shorter  edge:  for  example,  the  chain  in  Figure  6  goes  from  point 
1  to  point  2,  then  to  point  4  up  to  point  6.  This  last  should  be  linked  to  point  3, 
but  the  check  allows  to  verify  that  point  3  is  closer  to  point  2  (which  already 
belongs  to  the  chain),  thus  point  6  becomes  an  End  Point  and  the  link  2-3  is  retained. 

-  A  final  check  is  done  to  control  if  some  points  have  been  left  out  (this  might 
happen  to  points  close  to  a  Branch  Point)  and,  in  case,  they  are  linked  to  the  closer 
point  of  the  chain. 
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Figure  7  shows  the  result  of  the  chain  construction  in  the  first  frame  and  Table 
2  presents  the  values  relevant  to  this  chain,  as  they  are  output  by  the  computer 
program:  in  each  row,  the  first  two  values  are  the  labels  of  the  points  linked  and 
the  third  value  is  the  relevant  distance. 


FIGURE  7  -  CHAIN  OF  TARGETS  IN  THE  FIRST  IMAGE  OF  THE  SEQUENCE 


Counting  the  number  of  times  that  each  point  is  repeated  in  the  first  two 
columns,  the  algorithm  can  classify  the  hot-spots:  points  1,  6  and  8  are  End  Points 
(one  presence);  points  3,  4,  5,  7,  9,  10  and  11  are  Intermediate  Points  (two 
presences);  point  2  is  a  Branch  Point  (more  than  two  presences). 


DISTAM:  1  2  127.95 
DISTAM:  2  4  23.35 
DISTAM:  4  6  24.02 
DISTAM:  2  3  25.24 
DISTAM:  3  5  24.04 
DISTAM:  5  7  80.11 
DISTAM:  7  11  116.47 
DISTAM:  11  10  27.78 
DISTAM:  10  9  39.41 
D I S  f  AM :  9  8  34.21 

TABLE  2  -  LINKS  AND  DISTANCES  OF  THE  CHAIN  (first  image) 


Analyzing  the  values  of  the  last  column  of  Table  2,  we  have: 

/i(d)  =  52.26  , 

d(d)  =  38.57  , 

<J*(d)  =  10.45  ; 

being  d(d)  >  d*(d),  the  counter  K  is  initialized  and  the  differences 
(1)  |  d ( j  )  -  /i(d)  | 

are  compared  with  the  value: 

( 1+Kc )  •  <5(d)  =  (1+1-0. 2)-  38.57  =  46.29  . 

Two  differences  (1)  are  greater  than  this  last  value:  that  one  relevant  to  the 
link  of  point  1  with  point  2,  and  that  one  relevant  to  the  link  of  points  7  and  li . 
The  higher  value  corresponds  to  the  first  one,  thus  a  cut  is  produced  on  the  link  1-2 
and  point  1  is  rejected,  being  an  End  Point. 

The  analysis  proceeds  on  the  group  of  the  remainii  j  distances,  for  which  we  have: 
/i(d)  =  43.85  , 

d(d)  =  30.75  , 

<J*(d)  =  8.77  . 
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Again  a  cut  must  be  produced,  so  the  counter  K  is  incremented  and  the  differences 
(1)  are  evaluated  and  compared  with: 

(1+Kc)  •  cS(d)  =  43.05  . 

The  difference  relevant  to  the  link  7-11  is  greater  than  this  value:  the  chain  is 
thus  split,  obtaining  two  groups  to  be  analyzed. 

For  the  first  one,  composed  by  the  edges  that  link  points  number  2,  4,  6,  3,  5 
and  7,  we  have: 

ju(d)  =  35. 3S  , 

(5(d)  =  22.39  , 

(5*(d)  =  7.07  , 

K  =  3  , 

( 1+Kc )  •  (5(d)  =  35.82  . 

Evaluating  the  differences  (1),  it  can  be  easily  seen  that  the  link  5-7  must  be 
cut  and  point  7  eliminated  as  End  Point.  For  the  remaining  set  of  edges  (chain 
5-3-2-4-G)  we  have: 

p!d)  =  24.16  , 
did)  =  0.68  , 

<5*(d)  =  4.83  , 

and  the  chain  is  not  further  split. 

Analogously,  the  second  group  individualized  in  the  previous  step  (chain 
8-9-10-11)  is  not  further  subdivided,  being: 

p(d)  =  33.80  , 

d(d)  =  4.76  , 

(5*(d)  =  6.76  . 

The  result  „f  the  analysis  of  the  distances  in  the  first  frame  leads,  thus,  to 
the  location  of  a  formation  made  up  of  five  targets  (number  2,  3,  4,  5  and  6)  and  of 
a  formation  made  up  of  four  targets  (number  8,  9,  10  and  11). 

The  same  analysis  is  then  carried  on  considering  all  the  subsequent  frames  of  the 
sequence.  Figure  8  shows  the  chain  of  targets  obtained  in  the  last  image  and  Table  3, 
analogously  to  Table  2,  shows  the  relevant  links  and  distances. 


FIGURE  8  -  CHAIN  OF  TARGETS  IN  THE  LAST  IMAGE  OF  THE  SEQUENCE 


If  the  algorithm  is  applied  to  this  chain,  firstly  the  link  between  targets  6  and 
10  is  cut,  the  group  composed  by  targets  number  8,  9,  10  and  11  is  recognized  as  a 
formation  and  the  analysis  proceeds  or.  the  other  group  of  targets,  The  second 
iteration  cuts  the  link  5-7,  and  target  7  is  rejected  as  End  Point.  Link  1-2  is  cut 
in  the  third  iteration,  and  target  1  is  rejected.  Finally,  the  group  of  targets  2,  3, 
4,  5,  and  6  is  recognized  as  the  second  formation  of  the  frame. 
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The  Frequt  icy  Table,  filled  up  during  the  analysis  of  the  frames, 
existence  of  the  two  formations  in  the  sequence. 


confirms  the 


DISTAM! 

1 

2 

86.54 

DISTAMi 

2 

4 

19.10 

DISTAMi 

4 

3 

20.59 

DISTAM i 

3 

5 

26.91 

DISTAMi 

4 

6 

23.54 

DISTAMi 

6 

10 

115.87 

DISTAMi 

10 

11 

26.25 

DISTAMi 

10 

9 

36.25 

DISTAMi 

9 

8 

36.40 

DISTAMi 

5 

7 

113.30 

TABLE  3  -  LlilKS  AND  DISTANCES  OF  THE  CHAIN  (last  image) 


8.  CONCLUSIONS 

The  quantities  used  and  evaluated  and  the  comparisons  performed  during  the 
analysis  by  the  algorithm  presented  in  this  paper,  are  summarized  xn  Table  4. 


EVALUATED 

PREDEFINED 

PREDEFINED 

IMAGE 

SYSTEM 

CONSTANTS 

FEATURES 

FEATURES 

. Partial  Displacements 

•Maximum  Speed 

(Instantaneous  Speed) 

Value 

.Total  Displacement 
•Total  Displacement 

. standard 

Mean  Value 

— 

Deviation 

.Total  Displacement 

Threshold 

Standard  Deviation 

Value 

•Differences  of 

Total  Displacements 
and  Total  Displacement 
Mean  Value 

— 

... 

.Minimum  Distances 
.Minimum  Distances 

-Standard 

Mean  Value 

— 

Deviation 

•Minimum  Distances 

Threshold 

Standard  Deviation 

TO 

Value 

•Absolute  Value  of 

FE 

•Absolute 

Differences  of  Minimum 

Difference 

Distances  and  Minimum 

COMPARED 

Threshold 

Distances  Mean  Va)ue 

WITH 

Level 

-Sum  of  Weighted 

•Presence 

Presences 

Threshold 

Level 

.Mean  Value  of  Distances 

•Range  of 

in  the  Formation 

Acceptable 
Distance  Values 

•Distances  between  End 
Points  of  the  Formation 

•Preferable 

.Number  of  Branch  Points 

Formation 

— 

in  the  Formation 

Lay-Out 

•Number  of  Targets  in 
the  Formation 

.Number  of 
Munitions 

— 

.Angles  formed 

.Preferable 

by  Targets 

Formation 

— 

of  the  Formation 

Shape 

.Homing  Point 

— 

... 

TABLE  4  -  SUMMARY  OF  THE  EVALUATED  QUANTITIES  AND  PERFORMED  COMPARISONS 
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The  algorithm  has  been  tested  on  various  files,  in  order  to  tune  the  predefined 
parameters  and  to  check  the  behaviour  of  the  algorithm  itself. 

The  results  obtained  are  guite  satisfying:  thanks  to  the  number  of  comparisons 
and  controls  performed,  the  algorithm  can  reject  false  alarms  and  isolated  targets 
and  can  detect  the  true  formations  also  in  presence  of  a  large  number  of  points  in 
the  sequence . 

In  the  frame  by  frame  analysis,  a  wrong  grouping  could  be  achieved  in  case  of 
very  close  targets,  due  to  the  quantities  used  to  perform  the  comparisons  (i.e.  the 
mean  value,  the  standard  deviation  and  the  differences  (1)):  in  this  case  the 
standard  deviation  could  be  increased  by  the  short  distances  and  shortest  links  could 
be  cut,  since  the  differences  (1)  are  taken  with  their  absolute  value. 

Finally,  the  algorithm  works  very  well  in  the  evaluation  of  the  optimal  formation 
and  of  the  point  of  homing,  once  the  characteristics  of  the  weapon  system  have  been 
properly  set. 
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SUMMARY 

The  results  of  the  design  and  simulation  of  an  advanced  airborne  early  warning  (AEW)  system 
are  presented.  The  approach  is  based  on  modeling  operator's  reasoning  and  decision  processes  as 
well  as  battlefield  strategies.  The  tasks  are  divided  into  threat  assessment  and  tactical  planning. 
The  implementation  of  these  subsystems  is  carried  out  in  a  generic  expert-system  shell  developed 
specifically  for  this  purpose.  The  AEW  crew  is  provided  with  an  advanced  display  that  monitors 
all  transactions.  The  functionalities  of  this  prototype  AEW  system  are  demonstrated  using  an 
advanced  simulation  facility.  Simulation  results  show  that  decision  automation  can  be 
accomplished  in  real  time  and  can  prove  to  be  a  valuable  tool  in  an  airborne  early  warning 
environment. 


1.  INTRODUCTION 

Airborne  Early  Warning  (AEW)  systems  are  responsible  for  acquiring,  correlating,  and  tracking 
targets  using  radar,  identification  friend-or-foe  (IFF)  signals,  infrared,  and  other  passive  sensors.  They  are 
used  for  battlefield  management  and  often  serve  as  command,  control,  and  communication  (C3)  centers. 
Increasing  sophistication  of  vehicle  technology  and  performance  indicates  that  response  time  to  threats  will  be 
shorter  (due  to,  for  example,  stealth  technology).  At  the  same  time,  new  battlefield  concepts  imply  that  the 
number  of  decisions  and  tactics  will  be  higher  (due  to,  for  example,  combat  unit  dispersion).  Clearly,  these 
requirements  place  demands  beyond  the  capabilities  of  the  current  AEW  systems,  which  must  be  upgraded  to 
meet  the  challenges. 

The  tactical  decisions  made  by  the  AEW  crew  are  based  on  threat  levels  of  the  targets,  availability  of 
resources,  and  rules  of  engagement.  They  also  are  affected  by  geographic  location,  defense  condition,  and 
weather.  All  of  these  factors  are  subject  to  constant  changes  which,  if  not  taken  into  account  in  devising 
strategies,  can  lead  to  incorrect  decisions.  Furthermore,  the  crew  usually  have  to  keep  track  of  a  large 
number  (e.g.,  over  200)  of  targets.  This  situation  translates  to  an  overwhelming  amount  of  information  from 
which  relevant  data  are  extracted.  The  end  result  is  a  high-workload,  error-prone  environment  where 
correct  and  timely  decision  making  is  severely  handicapped. 

Previous  discussions  indicate  that  the  ciitical  issues  for  the  AEW  systems  are  information  management 
and  decision  making.  It  is  said  that  most  decision  failures  are  due  to  the  inability  to  extract  and  integrate 
relevant  data  in  an  effective  time-critical  manner  [1].  These  problems  can  be  best  addressed  by  decision 
automation.  This  approach  would  reduce  the  number  of  cases  to  be  dealt  with,  thus  allowing  the  AEW  crew 
to  concentrate  on  more  critical  cases,  and  thereby  improving  the  quality  of  decisions. 

Studies  have  been  made  in  the  past  in  applying  artificial  intelligence  to  battlefield  management, 
specifically  in  the  areas  of  threat  warning  [2j,  tactical  indications  [3],  tactical  situation  assessment  [4],  tactical 
decision  aids  [5],  and  others.  The  US  Air  Force  is  funding  the  Pilot  Associate  project  [6]  to  develop  a  cockpit 
information  management  and  decision-aid  system.  To  encompass  force  deployment  and  protection,  a  similar 
effort,  but  larger  in  scope,  is  being  carried  out  by  Grumman.  This  effort  is  concentrated  m  the  Crew 
Associate  project  [7,8],  which  is  being  developed  to  assist  the  AEW  crew  in  decision  making.  The  objective 
of  this  paper  is  to  describe  the  design  and  performance  of  a  real-time  automated  decision-making  system 
(ADEMS)  to  assist  operators  in  an  AEW  environment. 

ADEMS  is  an  expert-system  shell  designed  to  extract  information  such  as  location,  behavioral  pattern, 
electronic  surveillance  status,  and  other  relevant  data  to  establish  battlefield  strategies.  Decisions  are  made  by 
incorporating  resource  managemmt  techniques  and  short-  and  long-term  goals.  Typical  scenarios  would 
include  vector-intercept  (by  aircn.it),  patrol  assignment,  combat  unit  relief,  etc.  ADEMS  has  a  restricted 
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application  domain,  which  is  decision  making  within  the  AEW  environment,  although  it  can  be  expanded  to 
deal  with  other  problems. 

In  this  paper  we  describe  parts  of  Gnimman's  Crew  Associate  (Fig.  1).  It  consists  of  a  Threat 
Assessment  (TA)  module,  which  ascertains  the  threat  level  of  a  target;  a  Tactical  Planning  (TP)  module, 
which  makes  tactical  decisions  based  on  overall  battlefield  strategies;  a  Monitor  and  Control  System  (MCS), 
which  conducts  on-line  system  monitoring  of  AEW  aircraft  and  friendly  targets;  and  a  Crew  Console  Display 
(CCD),  which  provides  an  advanced  console  display  for  the  AEW  crew.  Both  TA  and  TP  are  implemented 
with  ADEMS.  Due  to  our  emphasis,  MCS  will  not  be  described  here.  The  work  described  herein  is  a 
continuation  of  previous  efforts  reported  in  Refs.  9  and  10. 

The  organization  of  the  paper  is  as  follows:  Section  2  describes  the  requirements  and  design  of  the  TA 
system,  while  Section  3  details  ;  talysis  and  design  of  the  TP  system.  Section  4  covers  the  internal  structure  of 
the  ADEMS,  with  emphasis  on  approach.  Section  5  describes  the  functionality  of  the  CCD.  Section  6  outlines 
an  advanced  simulation  environment  where  the  automated  AEW  functions  are  tested.  Section  7  presents  the 
simulation  results  using  typical  scenarios.  Finally,  Section  8  contains  conclusions  and  p'\ns  for  future  work. 


2.  THREAT  ASSESSMENT 

The  Threat  Assessment  (TA)  system  is  a  Crew  Associate  module  whose  function  is  to  advise  crews 
about  the  threat  levels  of  detected  targets.  TA  determines  a  target’s  threat  level  based  on  its  position 
(distance,  on/off  corridor,  etc.),  dynamic  behavior  (heading,  acceleration,  altitude,  etc.),  actions  (radar  on, 
jamming,  etc.),  identification  (F-14,  F-16,  enemy  aircraft,  friendly-known,  enemy-unknown,  etc.),  and  the 
number  of  aircraft  in  a  raid. 

Based  on  the  available  information,  the  TA  system  first  assigns  each  target  a  classification,  which  can  be 
confirmed-friend,  assumed-friend,  neutral,  unknown,  assumed-foe,  or  confirmed-foe.  This  information  may 
come  from  a  separate  identification  (ID)  system  or  can  be  inferred,  for  example,  from  the  target  flying 
pattern.  The  threat  level  is  then  assessed  from  a  combination  of  heuristic  parameters  such  as  actions,  distance, 
ID,  etc.  A  weighting  scheme  can  be  placed  on  each  of  the  parameters  to  achieve  desired  emphasis.  The  threat 
level  also  is  dependent  on  look-ahead  depth,  that  is,  how  much  TA  anticipates  enemy  moves,  thereby  putting 
different  "values"  on  incoming  targets.  The  heuristic  measures  are  added  to  arrive  at  a  single,  final  number 
from  which  the  threat  level  of  no-threat,  possible-threat,  or  high-threat  is  assigned.  More  specifically, 

Htotal  =  w,c-htc  +  wd*hd  +  wa*ha  +  wid»hid  +  wia*hia  (1) 

where  Htotal  is  the  final  score,  htc  is  the  heuristic  measure  associated  with  target  classification,  hd  is  the 
heuristic  measure  associated  with  distance,  ha  is  the  heuristic  measure  associated  with  action,  hid  the  heuristic 
measure  associated  with  ID,  hia  the  heuristic  measure  associated  with  look-ahead  depth,  and  w  are  appropriate 
weighting  factors.  A  realistic  application  of  heuristic  measures  can  be  found  in  [11], 

The  Threat  Assessment  module  is  implemented  using  ADEMS.  They  implement  most  of  the  rules 
described  in  (12);  examples  of  some  of  them  are  shown  in  Fig.  2.  The  output  of  the  TA  system  is  fed  to  the 
TP  module,  which  is  discussed  in  the  next  section. 


3.  TACTICAL  PLANNING 

The  Tactical  Planning  HP)  system  is  a  Crew  Associate  module  whose  function  is  to  recommend  (to  the 
AEW  crew)  or  implement  the  best  strategies  necessary  to  engage  threats  and  accomplish  a  desired  mission. 
This  is  closely  related  to  the  desired  combat  posture,  whether  defensive  or  offensive.  The  tactics  employed 
depend  on  the  threat  level  of  the  hostile  targets,  availability  of  resources,  the  rules  of  engagement,  defense 
conditions,  and  geographic  locations.  They  can  also  take  into  account  requirements,  such  as  positional 
advantage;  constraints,  such  as  guaranteeing  the  safety  of  the  carrier  group;  or  goals,  such  as  destroying  a 
particular  enemy  target. 

One  of  the  primary  functions  of  the  TP  system  is  to  assign  Combat  Air  Patrols  (CAPs),  tankers,  and 
Deck  Launch  Interceptors  (DLIs)  to  carry  out  a  mission.  This  task  is  carried  out  so  as  to  minimize  the 
exposure  of  the  carrier  group  to  danger  and  maximize  the  favorable  engagement  outcome.  TP  may  also  mean 
look  ahead  on  what  targets  may  become  a  high  threat  and  position  interceptors  in  advantageous  locations. 

The  other  important  role  of  the  TP  system  is  to  manage  resources  such  as  CAPs,  weapons,  tankers, 
jammers,  and  DLIs.  It  keeps  track  of  weapons  and  fuel  used  by  CAPs  and  ensures  their  availability  for  a 
successful  engagement.  It  also  advises  the  AEW  crew  of  unavailability  of  resources  and  recommends 
alternatives,  such  as  retraction  of  the  CAPs  in  self-defense. 
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Another  possible  usage  of  the  TP  system  is  to  devise  countermeasures  by  recognizing  enemy  plans  and 
tactics,  TP  can  also  incorporate  functions  that  anticipate  engagement  outcomes,  thus  revising  the  strategy  or 
the  overall  approach  (i.e.,  defensive,  offensive,  or  neutral)  accordingly.  These  notions  are  currently  under 
study. 


The  functionality  of  the  TP  system  reflects  a  number  of  generic  battlefield  management  rales.  Some  of 
them  are  shown  in  Fig.  3.  Note  that  although  both  TA  and  TP  rales  may  appear  to  be  simplistic,  they  become 
cumbersome  when  the  number  of  targets  is  large.  Furthermore,  the  combination  and  the  interaction  of  rales 
are  complex  and  are  better  addressed  using  decision  automation.  The  Tactical  Planning  module  is  also 
implemented  in  ADEMS,  which  is  described  in  the  next  section. 


4.  DESIGN  OF  AUTOMATED  DECISION-MAKING  SYSTEM 

Earlier  versions  of  the  TA/TP  systems  were  developed  using  the  Automated  Reasoning  Tool  (ART)  [7] 
and  the  Procedure  Reasoning  System  (PRS)  [9].  However,  the  processing  requirement  was  high  and  the 
system  response  was  sluggish.  It  was  felt  that  commercial  expert-system  shells  were  too  general  for  our 
purpose  and  too  slow  to  respond  to  the  inputs.  By  utilizing  these  shells,  we  are  effectively  (and  unnecessarily) 
paying  the  overhead  for  the  tools  that  are  superfluous.  In  an  effort  to  achieve  real-time  performance,  the 
functionalities  and  tasks  of  TA/TP  systems  were  carefully  examined  and  coded  as  LISP  procedures.  Although 
this  approach  was  successful  (10],  the  modularity  and  flexibility  of  the  design  were  sacrificed,  since  the 
inference  logic  had  to  be  "hard-wired."  Therefore,  it  was  decided  to  develop  a  generic  expert-system  shell 
(called  Automated  Decision  Making  System  or  ADEMS)  to  take  advantage  of  the  decision  structure  of  the 
AEW  scenario. 

The  desirable  features  of  the  ADEMS  do  not  differ  from  a  general  purpose  expert-system  shell.  They 
include  a  user-friendly  front-end  to  declare  and  to  modify  the  rales;  a  facility  to  display  relevant  and  current 
rales;  an  explanation  mechanism  to  describe  the  rules  invoked,  etc.  In  addition,  because  the  data  being  given 
to  the  TA/TP  systems  are  information  rich  and  carry  implicitly  the  preconditions  that  trigger  the  rules,  they 
naturally  lead  us  to  rely  heavily  on  data-driven  methods. 


The  data-driven  approach  is  used  here  not  in  its  "classical"  sense  of  forward-chaining,  but  rather  it  is  a 
reflection  of  the  decision  process.  It  is  accomplished  by  carefully  organizing  the  logic  and  scheduling  the 
execution  of  inference.  Thus,  specific  actions  are  associated  with  the  incoming  data,  and  they  are  embedded 
within  the  rales.  Each  action  consists  of  procedures,  messages  to  other  subsystems,  or  insertion  of  additional 
keywords  to  drive  the  inference  engine. 

A  major  step  in  setting  the  data-driven  structure  is  to  dissect  the  rale  entries.  ADEMS  uses  a  pseudo¬ 
parsing  system  to  understand  a  wide  variety  of  rale  specifications.  It  does  so  by  extracting  from  the  rule 
declaration  a  set  of  keywords  from  which  the  internal  logic  is  reconstructed.  Examples  of  keywords  are 
distance,  less-than,  greater-than,  equal-to,  CAP,  target,  high-threat,  low-threat,  corridor,  etc.  Any  input  that 
cannot  be  deciphered  in  this  fashion  is  redisplayed  and  the  user  is  prompted  for  clarification.  Extraction  of 
keywords  allows  ADEMS  to  decide  which  rules  should  be  invoked  without  use  of  any  meta-rales  (that  is, 
rules  about  usage  of  rules).  This  is  possible  since  ADEMS  knows  when  the  new  information  is  available,  and 
thus  only  those  rules  which  are  intersections  of  all  rales  will  be  executed.  Lastly,  any  rale  that  cannot  be 
parsed  (such  as  computation  of  heuristic  measures)  is  entered  as  a  special  case,  and  procedures  can  be  attached 
to  achieve  the  desirable  effects. 

After  the  rules  are  entered,  they  are  stored  on  the  property  lists  of  the  keywords.  This  is  true  whether 
the  rules  involve  actions  or  references  to  other  rules.  This  knowledge  representation  enables  ADEMS's 
inference  engine  to  quickly  access  the  desired  information  and  execute  the  required  procedures.  In  fact,  when 
new  data  become  available,  ADEMS  performs  a  simple  unification  of  all  existing  preconditions  and  runs  any 
rule  that  satisfies  the  prerequisites.  This  procedure  is  illustrated  with  some  examples  below. 

Rule  Structure 

IF  subject  +  verb  +  description  +  modifier 

THEN  action 

Example: 

IF  target's  ID  is  enemy  aircraft, 

THEN  the  target  is  an  assumed  foe 

Here  the  keywords  are  targetjd,  enemy_aircraft,  and  assumedjoe.  If  the  incoming  data  satisfy  the 
prerequisites,  an  "assumed_foe '  classification  will  be  associated  with  targetjd. 
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Example: 

IF  target  is  a  confirmed  foe 

AND  it  is  within  high-threat  zone, 

THEN  the  target  is  a  high  threat 

The  keywords  for  this  case  are  confirmed_foe,  high_threat_zone,  ard  high_threat.  When  the  first  two 
keywords  are  present  in  a  target's  property  list,  then  high  threat  is  declared  and  added  onto  the  target's 
property  list.  This  new  information  subsequently  will  trigger  rules  within  the  TP  system. 

Example: 

IF  target's  threat  level  is  high  threat 

THEN  assign  the  closest  CAP  to  engage  the  target 

All  keywords  are  previously  defined.  The  action  consists  of  a  procedure  that  figures  out  which 
available  CAP  is  closest  to  the  target  within  the  assigned  sector  and  then  vectors  it  to  intercept  the  target. 

The  approach  described  for  the  implementation  of  the  TA/TP  expert  systems  is  somewhat  narrow  in 
scope.  The  intent  is  to  achieve  a  good  compromise  between  fast  response  and  ease  of  use.  The  processing 
speed  is  gained  by  specializing  the  application,  and  the  flexibility  is  reduced  (but  not  lost)  by  concentrating  on 
specific  tasks.  Although  this  methodology  would  not  be  applicable  for  general  situations,  it  was  deemed  to  be 
sufficient  for  the  AEW  problem. 


5.  CREW  CONSOLE  DISPLAY 

The  purpose  of  the  CCD  is  to  display  the  current  combat  scenario  based  on  the  radar  and  other  sensor 
infoimation  to  an  AEW  operator.  It  also  communicates  to  the  crew  the  findings  of  TA  and  strategic  decisions 
made  by  TP.  Thus,  to  logically  display  the  current  situation  and  to  allow  quick  access  to  desired  information 
are  primary  goals  of  the  CCD.  A  very  large  part  of  the  CCD  design  is  human  interface.  This  is  necessary  so 
as  to  alleviate  the  operator's  workload  in  interpreting  information  and  making  queries  and  decisions. 


CCD's  design  is  graphic-intensive  so  as  to  simulate  a  conventional  radar  display,  but  with  several 
additions  such  as  pull-down  menus  and  mouse-sensitive  items.  Each  target  has  its  own  track  information 
(which  is  accessible  via  mouse  or  keyboard)  and  can  be  commanded  to  move  to  other  locations  in  a  similar 
manner.  Geographic  maps  are  overlaid  on  the  display  as  well  as  on  (simulated)  radar  sweep.  Zoom  and 
multiple  window  facility  are  available.  Color  is  judiciously  used  to  provide  information  differentiation.  The 
system  provides  automatic  hand-over  when  the  aircraft  move  out  of  the  assigned  sector  of  the  AEW  crew, 
thus  ensuring  proper  information  transfer. 


Threat  assessessment  and  subsequent  tactical  decisions  are  displayed  graphically  as  well  as  textua'.ly. 
The  AEW  operator  can  override  the  recommendation  made  by  TP  (and  TA).  Optionally,  the  decisions  by  TP 
can  be  made  to  take  effect  within  a  specified  time  interval.  Presently,  all  decisions  are  automatically 
transmitted  to  the  Simulation  Driver.  The  crew  has  additional  flexibility  in  changing  the  current  ID  of  the 
target  and  letting  the  TA/TP  expert  systems  evaluate  the  consequences.  This  leads  to  better  tradeoffs  of 
strategies. 

Although  the  CCD  is  an  integral  part  of  the  AEW  system,  its  detail  design  is  not  warranted  at  this  stage. 
However,  sufficient  flexibility  is  built  in  to  permit  future  improvements.  In  fact,  much  of  the  CCD  is 
modeled  after  the  Simulation  Driver,  which  is  to  be  described  in  the  next  section. 


6.  SIMULATION  ENVIRONMENT 

To  test  the  decision-making  process  of  the  TA/TP  modules,  we  have  developed  a  generic  Simulator 
Driver  (Fig.  1)  which  allows  us  to  set  up  and  experiment  with  various  combat  scenarios  quickly  and 
repetitively.  Moreover,  the  Simulation  Driver  would  allow  us  the  freedom  to  provide,  emulate,  and  encode 
necessary  data  to  other  modules  of  the  Crew  Associate. 

The  Simulation  Driver  is  designed  to  be  user  friendly  ar.d  complete.  Emphasis  is  placed  on  the  ability 
to  set  up  a  new  scenario  or  modify  an  existing  one  quickly  and  efficiently.  The  Simulation  Driver  currently 
supports  any  generic  scenario  with  arbitrary  user  specifications.  Targets  are  created  by  moving  the  mouse 
pointer  over  the  desired  location  and  clicking  the  appropriate  mouse  button.  They  are  displayed  according  to 
the  Naval  Tactical  Display  Symbol  (NTDS)  set,  which  includes  enemy  air  targets,  friendly  air  tracks, 
unknown  tracks,  surface  tracks,  etc.  Once  positioned,  a  target  can  be  deleted  or  dragged  to  ?.  new  location. 
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Targets  can  be  made  to  move  toward  or  away  from  fixed  points;  they  can  also  be  vectored  or  commanded  to 
escort  other  targets.  Target  color  can  be  changed  for  emphasis  or  bookkeeping. 

Associated  with  each  target  is  a  set  of  target  information  such  as  speed,  altitude,  heading,  target  ID,  etc. 
All  data  have  defaults  and  can  be  displayed  on  command.  Changes  to  data  can  be  made  individually  (via 
mouse)  or  to  all  targets  created.  Both  military  and  commercial  corridors  can  be  drawn  at  specified  locations. 
All  simulation  scenarios  and  sessions  can  be  recorded  and  saved  as  files  to  be  played  back  at  a  later  time  with 
different  parameters,  if  desired. 

An  impending  engagement  (between  two  targets)  is  highlighted  (with  a  circle  drawn  around  them). 
The  user  has  the  option  of  destroying  either  of  thr.  targets  or  selecting  the  "escort"  mode,  where  the  friendly 
target  changes  its  course  to  parallel  that  of  the  enemy  target.  Decisions  on  the  engagement  result  can  be  made 
automatically  (based  on  a  weighted  preassigned  probability)  or  entered  manually  by  an  operator.  Engagement 
results  are  transmitted  to  the  TA/TP  system  to  be  used  in  the  generation  of  subsequent  strategies. 

The  Simulation  Driver  is  written  in  LISP  in  object-oriented  format.  It  utilizes  various  advanced 
features  offered  by  the  Symbolics  expanded  Common-LISP  environment,  such  as  routines  that  keep  track  of 
the  mouse  and  generate  pull-down  menus.  The  database  for  the  targets  created  is  kept  using  generic  pointing 
structures,  which  allow  direct  reference  by  name  and  quick  access  to  the  data.  Currently,  the  Simulation 
Driver  resides  on  a  Symbolics  3765  and  is  displayed  on  a  Tektronix  color  monitor.  The  TA/TP  subsystems 
reside  on  a  Symbolics  Maclvory,  while  the  CCD  uses  a  separate  Symbolics  3765.  All  interactions  among  the 
TA/TP  modules,  the  CCD,  and  the  Simulation  Driver  are  coordinated  by  a  System  Manager  and  transferred 
over  Ethernet  using  proprietary  network  software. 

7.  SIMULATION  RESULTS 

Preliminary  evaluation  of  the  functionalities  of  the  Threat  Assessment  and  Tactical  Planning  modules 
are  shown  in  the  coordination  of  tactical  maneuvers  using  a  generic  combat  scenario  (Fig.  4).  The  objective 
is  the  defense  of  the  carrier  group  as  a  whole  (which  implies  maintenance  of  proper  defensive  posture). 
Targets  0-2  are  available  CAPs  and  targets  3-5  are  enemy  (or  hostile)  aircraft  targets.  Targets  6-7  are 
commercial  airlines  flying  in  defined  commercial  corridors.  Targets  8-12  are  friendly  and  unknown  surface 
tracks.  Target  13  is  the  (?  vehicle.  The  critical  distance  circles  (the  inner  one  is  the  high-threat  region  [with 
respect  to  the  carrier]  and  the  outer  one  is  the  minimum-threat  region)  are  shown  as  drawn.  The  goal  in  this 
case  is  to  protect  the  carrier  force  and  associated  ships.  The  sequence  of  events  is  as  follows: 

Event  1:  Targets  3-4  move  toward  the  carrier. 

Action  1:  High  threat  is  not  declared  since  none  of  the  enemy  targets  are  within  the  high-threat  region. 

Event  2:  Target  3  is  within  the  high-threat  region  and  its  radar  is  active. 

Action  2:  CAP-1  (or  Target  1),  which  is  the  closest  one  within  the  assigned  sector,  is  directed  to 
intercept  and  destroy  Target  3. 

Event  3:  CAP-1  destroys  Target  3  during  the  engagement.  Targets  4  and  5  are  still  minimum  threats. 

Action  3:  CAP-1  becomes  an  available  CAP. 

The  new  scenario  is  shown  in  Fig.  5. 

Event  4:  Targe:  4  crosses  the  high-threat  region,  but  its  radar  is  inactive  and  it  is  flying  at  a  lower 
speed;  therefore,  it  is  considered  a  minimum  threat.  At  the  same  time  Target  7  (an  assumed 
commercial  airliner)  deviates  from  the  corridor  and  heads  toward  the  friendly  surface  ships 
(Target  8). 

Action  4:  CAP-1,  again  being  closest  to  Target  4  within  the  sector,  is  assigned  to  escort  Target  4. 
CAP-0  is  repositioned  closer  to  the  surface  ships  as  a  precautionary  measure. 

The  new  scenario  is  shown  in  Fig.  6. 

Event  5:  Fuel  on  CAP-1  is  running  low  and  Target  7  moves  within  the  high-threat  region  of  the 
surface  ships. 

Action  5:  CAP-2  is  assigned  to  replace  CAP-1,  which  returns  to  carrier  for  refueling,  while  CAP-0  is 
commanded  to  leave  the  patrol  mode  and  intercept  Target  7. 

Event  6:  Target  7  is  verified  to  be  an  enemy  airplane  and  is  destroyed  in  the  engagement.  Target  4 
moves  away  from  the  carrier  force. 

Action  6:  Both  CAP-0  and  CAP-2  become  available. 

This  final  scenario  is  shown  in  Fig.  7. 

Other  combat  scenarios  have  been  tested.  They  show  that  all  decisions  can  be  made  essentially  in  real 
time.  This  performance  is  mostly  due  to  coding  efficiency  and  partially  due  to  scenarios  used  and  strategies 
involved.  It  remains  to  be  seen  what  performance  will  be  achieved  when  more  sophisticated  tactics  are  used 
and  operators  become  part  of  the  overall  system. 
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8.  CONCLUSIONS 

Design  and  simulation  of  an  advanced  airborne  eaily  warning  system  have  been  described.  The  work  is 
carried  out  by  applying  automated  decision-making  techniques,  which  emulate  the  inference  process  of  the 
AEW  operators.  The  tasks  are  separated  into  threat  assessement,  which  determines  a  target's  threat  level 
based  on  available  information,  and  tactical  planning,  which  attempts  to  use  the  available  strategies  to  achieve 
the  optimal  response.  Both  TA/TP  modules  are  implemented  using  a  generic  expert-system  shell  designed 
specifically  for  the  AEW  problems.  The  shell  is  based  on  data-driven  concepts  and  provides  necessary 
knowledge  representation  and  inference  engine  to  ease  the  design.  Additionally,  an  advanced  crew  console 
display  and  a  unique  simulation  facility  have  been  developed  to  further  evaluate  the  potential  of  the 
knowledge-based  approach.  The  systems  are  being  integrated  and  tested  against  several  scenarios,  and  the 
results  are  encouraging. 

Future  plans  call  for  refinements  of  current  tactics  and  the  expert-system  shell.  We  plan  to  conduct 
experiments  using  real  data  from  an  E-2C  Hawkeye.  We  also  contemplate  research  on  decision  making  based 
on  partial  data,  which  is  due  to,  for  example,  radar  jamming.  In  this  situation,  it  would  be  necessary  to  rely 
on  the  outputs  of  secondary  sensors  and  carry  out  data  fusion  in  both  the  identification  stage  as  well  as  in  the 
inference  process.  Finally,  we  are  looking  at  potential  applications  to  programs  such  as  Joint  Surveillance  and 
Tracking  System  (J-STARS),  ground  monitoring,  and  with  some  modifications,  to  the  air-traffic  control 
systems. 
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IF  the  threat  level  of  a  target  is  high  threat, 

THEN  assign  the  closest  available  CAP  to  engage  with  the  target. 

IF  a  high-threat  target  has  been  destroyed, 

THEN  make  assigned  CAP  available  and  update  information. 

IF  there  are  no  available  CAPs  in  the  area, 

THEN  dispatch  a  combat-ready  CAP  from  the  carrier. 

IF  the  CAP  available  for  assignment  is  low  in  fuel  and  a  tanker  is  available, 
THEN  refuel  the  CAP  in  air. 


Fig.  3  Some  Tactical  Planning  Rules. 


Fig.  7  AEW  Scenario  after  Action  5. 


Fig.  8  Final  AEW  Scenario. 
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Cette  communication  fait  suite  aux  travaux  rdalisPs  dans  le  cadre  d'un  marche  S.T.T.E 


RCsumb  :  La  technoiogie  des  dquipements  d'avion  de  combat  est  en  constante 
Evolution  L'augmentation  de  la  precision  des  instruments  de  navigation, 
de  la  puissance  des  calculateurs,  du  nombre  de  moyens  de  recalage 
amdliore  la  capacity  de  rAussir  les  missions  modernes.  Cependant,  la 
tache  du  navigateur  s'en  trouve  compliqube.  SEAN  (Systdme  Expert 
d'Aide  &  la  Navigation  pour  avion  de  combat)  permet  d'assister  le 
navigateur  en  le  conseillant  dans  ses  choix,  tout  en  surveillant  le 
comportement  du  systdme  de  navigation.  L’emploi  d’une  mbthodologie 
cognitive  performante  (KOD)  associde  A  un  pnncipe  robuste  de  validation 
des  connaissances  a  permis  de  dbvelopper  de  (agon  cohSrente  une 
maquette  trds  complete  du  syst&me  expert  et  de  son  environnement 
embarqud  opdrationnel. 

Mots  elds  :  SystCme  Expert,  Navigation  Inertielle,  Diagnostic  Technique. 


Abstract  :  Technology  of  fighter  aircrafts  equipments  is  in  constant  evolution 
Increases  in  precision  of  navigation  equipments,  in  number  of  available 
means  of  navigation  updating  and  in  onboard  computers  performance,  has 
improved  aircraft  capability  to  satisfy  modem  missions  requirements  but 
also  has  complicated  copilot's  tasks.  SEAN  (Expert  System  for  Navigation 
Aiding  aboard  fighters  has  been  designed  to  assist  copilot  in  its  choices 
and  supervise  the  inertial  hybrid  navigation  system  Use  of  KOD 
(Knowledge  Oriented  Design),  a  powerful  cognitive  methodology, 
associated  to  robust  knowledge  validation  principles,  has  allowed 
development  of  a  prototype  of  the  expert  system  working  with  a  realistic 
complete  simulation  of  its  onboard  environment. 

Key-words  Expert  System,  Inertial  Navigation,  Technical  Diagnosis 
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0  INTRODUCTION 


Le  systdme  de  navigation  inertielle  consider  dans  le  cadre  de  cette  etude  se  compose 
de  deux  UNI  (Unites  de  Navigation  Inertielle)  et  dispose  de  quatre  moyens  de  recalage 
de  navigation  associds.  Chaque  unite  de  navigation  inertielle  peut  fourmr  deux  types 
d'information  de  navigation  : 

-  une  navigation  inertielle  recalls  classiquement, 

-  une  navigation  inertielle  optimale  i.e.  recaiee  par  filtrage  de  Kalman. 

Le  navigateur  assure  la  mise  en  oeuvre  du  systems  et  realise  des  choix  qui 
conditionnent  I'obtention  d'une  bonne  performance  de  navigation  : 

•  selection  de  la  meilleure  navigation  (une  parmi  quatre), 

-  choix  d'un  moyen  de  recalage  (un  parmi  quatre), 

-  validation  ou  non  des  recalages. 

Ces  choix  son,  assez  cntiques  et  le  navigateur  ne  dispose  pas  toujours  du  temps  et  des 
moyens  de  reflexion  suffisants  pour  assurer  la  pnse  de  decision  optimale.  Un  systeme 
expert  a  done  6\6  envisage  aim  de  I'assister  et  de  permettre  la  mise  en  oeuvre  optimale 
du  systems  de  navigation.  II  comporte  deux  volets  : 

•  une  surveillance  constants  des  equipements  pour  avoir  k  chaque  instant  une  idee 
qualitative  de  leur  fonctionnement, 

■  une  aide  au  copilots  consistant  en  des  prises  de  decision  bas6es  sjr  la 
connaissance  precise  de  ret  at  du  systems. 

Un  avantage  par  rapport  au  navigateur  est  de  pouvoir  traiter  I’ensemble  des 
informations  internes  de  la  navigation  et  de  pouvoir  y  appliquer  une  expertise  de 
sp6cialistes  fconcepteurs  et  exp6rimentateurs  du  produit). 

Plus  pr6cis6ment,  I'apport  d'un  tel  systems  est  sigmficatif  k  trois  niveaux  : 

•  aide  k  la  pnse  de  decision  du  navigateur,  notamment  dans  les  cas  difficiles, 

-  amelioration  des  performances  du  systems  de  navigation  grace  k  Sexploitation 
des  redondances, 

-  surveillance  constante  des  equipements  permettant  une  maintenance  preventive 
plus  efficace. 

Comme  indiqu6  ci-dessus,  ce  travail  a  6te  realise  dans  I'hypothese  d'un  equipage  k 
deux  comprenant  un  copilots  charge  de  la  navigation.  Cependant,  la  mise  en  oeuvre 
d'un  tel  systems  a  bord  d'un  avion  de  combat  monoplace  deviendrait  une  necossite 

L'6tude  presentde  dans  cet  article  a  done  consiste  en  la  realisation  d'une  maquette 
informatique  mettant  en  couvre  une  simulation  realists  de  I'environnement  operationnel 
d'un  avion  de  combat. 


1  METHODOLOGIE  ET  DEROULEMENT  DE  L'ETUDE 


1.1  INTRODUCTION 

L'etude  a  comports  deux  phases  principales  : 

-  recueil  intensif  des  connaissances, 

-  realisation  de  la  maquette  de  demonstration. 

Toutelois,  au  dele  de  ce  ddcoupage  primaire,  c'est  un  long  travail  methodique  qui  a 
permis  I'aboutissement  de  cette  maquette  du  systeme  expert.  Afm  de  bien  maitriser 
les  aspects  cognitifs  de  l’6tude  du  systems,  deux  m6thodes  ont  6t6  mises  en  place. 

Tout  d'abord  nous  avons  retenu,  parmi  les  methodes  de  specification  et  de 
conception  de  logiciel,  une  methods  propre  au  genie  cogmtif  et  non  simplement  du 
domains  du  g6nie  logiciel  (m6thodes  type  SADT,  OOD,  ...)  II  s’agit  done 
veritablement  d'une  methods  d'acquisition  et  mod6lisation  des  connaissances 

En  complement,  une  methods  de  gestion  globale  de  l'6tude  a  6t6  eiaborde  visant  £ 
valider  les  connaissances,  le  plus  tot  possible  dans  le  cycle  de  vie  de  I'application. 
Ces  deux  aspects  m6thodologiques  sont  devetoppes  plus  precis6ment  dans  les 
paragraphes  qui  suivent. 


1.2  METHODE. D'ACQUISITION  ET  DE  MODELISATION  DES.  CONNAISSANCES 

Le  recueil  et  la  validation  des  connaissances  ont  ete  realises  avec  la  methods  KOD 
tirde  des  travaux  de  Claude  VOGEL  [VOG  88],  Celle-ci  s'appuie  principalement  sur 
deux  iddes  : 

-  la  decomposition  structurale  des  objets  mampulds  par  I'expert  et  des  fonctions 
auxqueiies  ils  sont  assujettis, 

-  I'association  de  I'information  aux  conditions  de  I'dnonciation  ;  on  conserve  ainsi 
une  connaissance  impr6gn6e  de  la  vision  de  son  enonciateur 

Pour  la  presenter  rapidement,  disons  qu'elle  consisto  £  dlaborer  trois  moddles 
successes  permettant  de  passer  du  discours  de  I'expert  £  la  realisation  du  systems 
expert  (voir  figure  1.1) : 

-  le  models  pratique  -  Cette  phase  a  pour  but  d'identifier  et  do  specifier  I'expertisu 
On  part  d'une  retranscription  fideie  des  propos  de  I'expert  de  laquelle  on  extrait 
les  concepts  mam6s  par  ce  dernier,  les  actions  s’exergant  sur  ces  concepts  et 
enfin  la  lists  des  inferences  naturellement  exprimees  On  obtiont  ainsi  une  Dase 
d'6nonc6s. 

-  le  mod£le  cognltif  -  Cette  6tape  essentielle  conduit  £  l’6laboration  d'un  models 
abstrait  intermediate  permettant  de  reproduire  les  raisonnements  profonds  de 
I'expert  et  non  pas  seulement  une  m£canique  de  resolution  dependants  des  cas 
6ludi6s.  Ce  models,  ind6pendant  de  touts  representation  informatique,  est  eiabore 
par  structuration  de  la  base  d'enonc6s  obtenue  pr6c6demment  en  classifications 
(taxinomies),  actions  generiques  (actmomies)  s'exergant  sur  ces  classifications  et 
schemas  d’inference. 

-  le  mod£le  informatique  -  C'est  la  transcription  du  models  cognitif  dans  un 
langage  informatique  (incluant  le  plus  souvent  une  representation  objet,  un 
langage  £  base  o'e  regies  et  un  langage  procedural). 

La  volonte  d'appliquer  une  methods  apparemment  aussi  contraignante,  part  d'un 
constat  d£sormais  classique  :  it  est  n£cessaire  d'6tabhr  au  cours  du  processus 
d'acquisition  de  la  connaissance  un  models  abstrait  interm6diaire  afin  de  reproduire 
les  raisonnements  profonds  de  I'expert  e.  non  pas  seulement  une  mdcanique  de 
resolution  dependants  des  cas  £tudi6s. 

En  effet,  lorsque  cette  etape  est  respect£e,  on  obtient  des  syst£mes  £  base  de 
connaissance  qui,  bien  que  Iimit6s  dans  leurs  capacit6s  de  raisonnement, 
conduisent  toujours  les  expertises  £  I'image  de  celles  que  r£aliserait  I'expert. 
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DISCOURS  PERCEPTION  DU  MONDE  REEL 


MODELE 

PRATIQUE 


BASE  D’ENONCES 


MODELE 

COGNITIF 


MODELE 

INFORMATIQUE 


Figure  1.2  METHODE  KOD  (Knowledge  Oriented  Design) 


Inglntaifs 


Figure  1.1  CYCLE  DE  DEVFLOPPEMENT 

1.3  APPROCHE  GLOBALE  :  VALIDATION  DES  CQNNA1SSANCES 
1.3.1  Ip'  .uction 

Dans  le  d6veloppement  d’un  systdme  expert,  de  rr>5me  que  dans  tout  projet 
informatique,  il  y  a  lieu  de  minimiser  le  travail  de  mise  au  point.  Pour  cela,  une  bonne 
specification  des  fonctions  attendues  du  systdme  est  n^cessaire,  mais  de  m§me  que 
I’on  effectue  des  test  unitaires  de  parties  de  logiciel,  il  y  a  dgalement  lieu  de  valider 
ia  connaissance  au  fur  et  a  mesure  de  son  recueii. 


De  plus,  nous  disposons  E  la  SAGEM  de  logiciels  permettant  de  simuler  trds 
prEcisEment  le  comportement  d'une  UNI.  On  peut  ainsi  reproduce  Involution 
inertielle  d'un  avion  en  fixant  la  trajectoire  et  I'Etat  des  composants  inertiels  des 
UNIs. 

Ce  simulateur  nous  a  permis  d'envisager  un  processus  de  validation  en  deux  temps 
(voir  figure  1 .2). 

1,3,2  Validation  amont 

Dans  notre  application,  nous  avons  confronts,  des  leur  recueil,  les  propos  tenus  par 
les  diffdrents  experts  aux  phEnomEnes  observables  sur  des  rEsultats  de  simulations 
inertielles. 

Nous  ne  commencions  E  Elaborer  le  module  pratique  que  lorsque  tous  les  propos 
tenus  par  I'expert  Etaient  cohErents  avec  les  Evolutions  constatEes  sur  les  rEsultats 
de  simulations. 

Nous  avons  pu  ainsi  Eliminer  les  expertises  incertaines  et  favoriser  I'expression  de 
nouvelles  connaissances. 

13,3  Yalidatiflii  aval 

Les  mEmes  simulations  ont  Egalement  EtE  utillsEes  aprEs  chaque  passe 
d'implEmentation  afin  de  vEnfier  la  cohErence  des  connaissances  introduites  dans 
la  maquette  informatique. 

Enfm,  ©lies  nous  ont  servi  de  base  pour  la  validation  finale  de  la  maquette  dans  sa 
version  stabilisEe  actuelle. 


2  CONNAISSANCES  ET  ARCHITECTURE  DU  SYSTEME 


2.1  INTRODUCTION 

Afin  de  Wndficier  d'une  modeiisatlon  du  domains  traitb  aussi  complete  qua 

possible,  nous  avons  fait  appel  k  plusieurs  experts  : 

-  les  ingbnleurs  delude  en  systemes  inertiels  (Ing6nleurs  "materiel”  et  'logiciel")  qui 
ont  acquis  une  experience  k  travers  les  etudes  de  conception  nt  la  misa  au  point 
de  systemes  inertiels  hybrides,  les  simulations  de  performances  et  6galement 
i'analyse  des  resultats  d'essais  en  vol.  Its  ont  done  fourni  une  connaissance  qua 
I'on  peut  qualifier  de  profonde. 

--  de  m6me,  les  ing6nieurs  d'essais  en  vol  ayant  participb  &  la  mise  au  point  des 
systemes  qui  ont  donne  leur  vision  plus  experimental  du  comportement  des 
UNIs.  Ce  sont  eux  qui  ont  recens^  cerfaines  anomalies  ou  derives  suspectes 
sutvenues  an  cours  de  vols  d'essais. 

-  enfin,  les  pilotes  /  navigateurs  d'essais  des  Centres  d'Essais  en  Vol  qui  ont  plutot 
raisonne  en  terme  de  "boite  noire”.  C'est  done  plus  une  connaissance  de  surface 
sur  I'utilisation  du  systems  de  navigation  qui  a  ete  extraite. 


2.2  OOMAINES  DE  CONNAISSANCES 

Au  cours  des  difierents  entretiens,  nous  avons  mis  en  evidence  4  prmcipaux 

domaines  de  connaissances  : 

•  connaissances  sur  les  systemes  inertiels  •  sources  d'erreurs,  leur  ordre 
d'importance,  leur  influence  sur  les  sorties  de  navigation,  leur  frequence 
d'apparition, 

-  connaissances  sur  les  moyens  de  recalage  :  precision  a  prion,  influence  sur  les 
sorties  de  navigation,  leur  frequence  d'appantion, 

-  connaissances  sur  le  filtrage  de  Kalman  .  sources  d'erreurs,  leur  ordre 
d'importance,  leur  influence  sur  les  sorties  de  navigation,  leur  frequence 
d'appantion, 

•  experience  du  navigateur 


2.3  REPRESENTATION  DES  CONNAISSANCES 

2.3,l.lntrcductiafl 

Nous  n'avons  pas  cherche  e  ce  stade  k  mnover  au  travers  d’une  architecture  ou 
d'une  technique  d6veloppee  specifiquement  pour  notre  problems.  Notre  but  k 
travers  cette  etude  n'etait  pas  de  concevoir  un  nouvel  outil  mais  bel  et  bien 
d'appliquer  des  techniques  rodbes  e  I’aide  d'un  produit  du  marche. 

Aussi,  suite  k  I’eiaboraticn  du  models  cognitif,  nous  avons  simplement  dressb 
I'inventaire  des  modes  de  representations  ndeessaires  pour  implbmenter  la  base  de 
connaissances  : 

-  tout  d'abord,  une  representation  centres  objet  Celle-ci  permet  de  poser  la 
structure  de  la  base  de  connaissance  en  donnant  un  canevas  sur  lequel 
s’appuie  et  se  developpe  le  raisonnement  de  ('expert  . 

-  pour  completer  cette  representation,  des  actions  qui  sont  des  morceaux  de 
code  proceduraux  rattach6s  aux  objets.  Celles-ci  peuvent  etre  executes,  soit 
e  la  demands  (m6thodes  appelbes  en  partie  droite  de  regies),  soil 
automatiquement  lors  de  I'accbs  ou  plus  genbralement  la  manipulation  d'un 
attnbut  d'un  objet  (demons). 

-  les  rdgles  permettant  de  batir  la  logique  du  raisonnement 

-  enfin,  pour  permettre  le  raisonnement  k  I'int6rieur  de  mondes  clos  ou  pour 
gen6rer  des  hypotheses,  le  mecanlsme  de  contextes. 
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2.3.2  La  structure  centre  obiat 

Suite  £  I'analyse  du  discours  des  experts,  quatre  classifications  principals  ont 

Pmerg£ : 

-  Les  sources  d'erreurs  Inertieiles.  Celles-ci  se  dPcomposent  en  deux 
categories  :  les  sources  d'erreurs  entrainant  des  erreurs  visibles  £  long  terme  et 
ceiles  dont  les  erreurs  sont  observables  4  court  terme.  Au  niveau  le  plus  bas  de 
cette  taxinomie,  on  retrouve  des  decalibrations  de  facteurs  d'echelles  ou  de 
biais  de  composants  inertiels.  N6anmoins,  il  ne  s'agit  pas  (,'une  decomposition 
structured  complete  d'une  UNI.  Seules  les  sources  d'erreurs  slgnlficatives  ont 
ete  conservbes. 

-  Les  PvPnements.  II  est  apparu  rapldement  que  parmi  toutes  les  donnPes 
inertieiles  possibles  en  entree  du  systems  expert,  certaines  etaient  plus 
Interessantes  £  observer  que  d'autres  lorsque  surviennent  au  cours  du  vol  des 
evenements  particuliers  qui  excltent  la  dynamique  de  I'avion.  Les  virages, 
changements  d'altitude  et  alignement  constituent  de  tels  evenements. 

-  L'historlque.  Celui-ci  est  une  collection  de  toutes  les  tendances  qui  ont  pu  etre 
enreglstr6es  au  cours  des  vols  precedents  (erreurs  de  vitesse  au  retour 
parking,  ...  etc).  II  constitue  la  memoirs  du  comportement  £  long  terme  de 
chaque  UNI. 

-  La  description  fonctionnelle  du  systems  de  navigation  et  de  ses  paramPtres 

observables  (longitude  pure,  longitude  recalbe . ). 

2.3.3  Les  actions 

On  a  distingue  quelques  types  principaux  d'actions  £  rpaliser : 

•  tout  d'abord,  un  ensemble  de  programmes  qui  permettent  de  d£tecter  I'arnv6e 
d'6v6nements  et  de  b£tir  les  structures  "objet"  nPcessaires  pour  les 
repr6senter, 

-  ensulte  des  attachements  proceduraux  qui  interviennent  lors  de  la  creation  des 
evenements  d'une  part  pour  en  renseigner  tous  les  champs,  d'autre  part  pour 
en  extraire,  le  cas  6ch6ant,  des  indices  interessants  pour  alimenter  la  logique 
de  resolution, 

-  enfin,  des  actions  cod6es  directement  en  parties  droites  de  r6gles  qui 
permettent  de  g6rer  les  m6canismes  d’hypoth£ses  tant  pour  les  causes 
d'erreurs  inertieiles  que  pour  les  recalages. 

2.3.4  Les  regies 

On  distingue  deux  ensembles  principaux  de  rpgles  assurant  des  parties  distinctes 
du  mPcanlsme  de  raisonnement. 

•  des  rpgles  chargpes  de  trailer  les  recalages.  Celles-ci  se  chargent  de  valider 
ou  de  ne  pas  valider  les  recalages,  en  lieu  et  place  du  navigateur. 

-  des  r6gles  chargees  de  prendre  en  compte  les  erreurs  inertieiles  ainsi  que  leurs 
interactions  avec  les  recalages.  Celles-ci  interviennent  en  association  avec 
des  mPcanismes  de  raisonnement  plus  complexes  que  sont  les  contextes. 

2A5  Les.contgxtes 

La  forme  de  la  connaissance,  telle  qu'elle  a  6t6  exprimpe  par  les  experts,  nous  a 
conduit  £  envisager  I'utilisation  de  contextes.  En  effet,  elle  laisse  apparaftre,  entre 
autres,  la  nPcessItP  de  I'emplol  de  mPcanismes  de  raisonnements  hypothPtico- 
dPductifs.  Un  tel  mPcanisme  peut  se  dPcnre  par  le  schema  suivant : 


Pour  impIPmenter  de  (agon  efficace  ces  mPcanismes,  une  solution  pertinents  est 
I'emploi  d'un  contexte  pour  chaque  hypothPse  gPnPrPe  tant  que  celle-ci  n’est  pas 
rPfutPe. 
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Nous  avons  distingue  pour  I’instant  deux  niveaux  principaux  de  raisonnement : 

-  la  recherche  de  causes  d'erreurs  inertielles.  Supposons  que  I’on  constate  une 
anomalie,  celle-ci  ayant  deux  causes  d'erreurs  possibles.  On  g6nere  alors 
deux  hypotheses  et  tous  les  6v6nements  d6tect6s  par  le  systems  sont 
interpretes  en  parallels  dans  les  deux  contextes  jusqu'e  ce  qu'un  6v6nement 
puisse  prouver  ou  retuter  une  de  ces  deux  hypotheses. 

-  la. determination  de  la  conliance  aue  Ton  oeut  accorder  e  un  recalaqe  (a 
posteriori).  Dans  les  systemes  classlques  la  decision  de  validation  d'un 
recalage  est  dlaboree  e  partir  d'une  Image  instantanee  donnee  par  le  filtre  de 
KALMAN.  Dans  notre  systems,  nous  utilisons  toutes  les  donndes  inertielles 
disponibles  depute  le  debut  du  vol  pour  decider  si  le  recalage  est  validable  ou 
non.  Cependant,  lorsque  nous  avons  un  doute  sur  la  qualite  du  recalage,  nous 
gen6rons  deux  hypotheses  concurrentes  (recalage  correct  /  recalage  mauvais) 
et  nous  attendons  I'arrivee  de  nouveaux  ev6nements  pour  determiner  a 
posteriori  la  contiance  que  Ton  peut  accorder  au  recalage. 

Ces  deux  niveaux  de  fonctionnement  ne  sont  cependant  pas  d6corr6l6s.  En  effet,  la 
detection  de  certains  phenomenes  anormaux  apres  un  recalage  peut  deboucher  sur 
deux  interpretations : 

-  la  presence  d'une  erreur  inertielle  sur  un  systems  bien  recald, 

-  un  mauvais  fonctionnement  dO  e  un  recalage  de  qualite  moyenne. 

■I  est  done  necessaire  d'avoir  une  vision  differente  des  erreurs  inertielles  selon  que 
le  recalage  est  consider  comma  bon  ou  mauvais 

L'impiementation  d'une  telle  structure  e  deux  niveaux  a  6t6  facilitee  par  I'existence 
du  mdcanisme  de  "View ■points’’  multiniveaux  sur  I’outil  utilise  (ART™). 


2.4  ARCHITECTURE  GLOBALE  DU  SYSTEME  EXPERT 

La  figure  2.1  presente  I'architecture  globale  tr&s  classique  de  la  maquette  du 
systems  expert.  Elle  fait  apparaitre  plusieurs  modules  importants  : 

-  les  informations  inertielles  en  provenances  des  deux  UNI  sont  lues  par  un 
module  d'acqulsltlon  de  donnees.  Celui-ci  est  egalement  charge  de 
transformer  les  donnees  brutes  en  faits  acceptables  par  la  base  de  faits  qui  est 
structures  en  objets, 

-  la  base  de  connalssances  contient  toutes  les  regies  et  les  methodes 
ndcessaires  au  raisonnement  du  systems, 

-  le  mecanisme  d'lnfdrence  seioctionne,  en  fonction  des  faits  existants,  des 
r6gles  ou  des  methodes.  II  active  les  regies  ou  execute  les  methodes,  le  resultat 
dtant  renvoyd  dans  la  base  de  faits.  II  gdre  dgalement  plusieurs  hypotheses  en 
paralieie  en  maintenant  la  coherence  de  la  base  de  faits, 

•  enfm  I'interface  utlllsateur  permet  d'observer  les  conclusions  du  systems 
expert  ou  de  lui  fournir  les  informations  suppl6mentaires  (avis  sur  la  valide'ion 
du  recalage  par  exemple) 


3  REALISATION  INFORMATIQUE 


La  maquette  de  demonstration  a  6t6  realises  pour  montrer  les  ameliorations  qua  peut 
apporter  le  Systems  Expert  pour  r6soudre  les  ditficultes  rencontrees  dans  un  oas 
d'utilisation  operationnelle.  Pour  cela,  le  Systems  Expert  a  ete  integr4  dans  un 
environnement  simulant  le  fonctionnement  des  systdmes  aveo  lesquels  il  sera 
appeie  e  traveller :  les  deux  UNI  et  les  instruments  du  poste  de  navigation. 

Cette  maquette  r4unit  les  fonctionnalites  suivantes  (voir  figure  3.1): 

-  possibility  de  pilotage  de  la  trajecto^e  k  I'aide  d'une  souris  avec  visualisation 
de  type  "fete  Haute"  de  I'image  syntliutique  du  MNT  correspondant, 

-  simulation  synchrone  des  deux  systemes  inertiels  hybrides,  ainsi  que  des 
moyens  de  recalage, 

-  simulation  du  poste  de  commands  et  de  navigation  PCN  et  du  poste  de 
selection  de  modes  PSM  afin  de  permettro  la  visualisation  de  divers  parametres 
et  la  prise  en  comote  des  recalages, 

-  aide  k  la  navigation  par  I'mterm6diaiie  du  Systems  Expert, 

-  moyens  de  d6pouillement. 

Ces  fonctionnalites  on!  4t6  impiementees  en  deux  parties  : 

-  simulateur  pllotable  de  navigation  Inertielle,  realise  sur  mmi-systeme  HP1000 
A-900, 

-  Systems  Expert  d'Aide  k  la  Navigation  et  simulation  des  equipements 
PCN  &  PSM,  realises  sur  machine  LISP  SYMBOLICS  avec  le  g6n6rateur  de 
svst6mes  experts  ART™  de  INFERENCE  Corp 

La  maquette  est  constituee  de  neuf  processus  asynchrones  fonctionnant  en  pseudo 
temps  r6el.  Le  systems  expert  et  le  simulateur  des  boitiers  PSM/PCN  sont 
impiementes  sous  la  forme  de  deux  processus  communiquant  par  une  liaison  de  type 
"producteur/consommateur"  :  un  premier  processus  fait  ('acquisition  des  donnees, 
soit  depuis  un  fichier,  soil  depuis  une  simulation  inertielle  et  les  envois  k  un 
processus  ART™. 
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4  RESULTATS  OBTENUS 


Afin  de  construire  un  ensemble  coherent  dans  le  temps  imparti  pour  l'6tude,  nous  avons 
limitd  le  nombre  de  causes  d'erreurs  prises  en  compte. 

Nous  avons  traits  complement  le  diagnostic  des  six  plus  importantes  causes  d'erreurs 
ainsi  que  leurs  correlations  avec  la  qualitS  des  recalages  vatidAs. 

Les  bases  de  connaissances  obtenues  ont  StS  testAes  sur  une  dizame  de  simulations. 

Plusieurs  rSsultats  ont  StS  obtenus  : 

-  d'une  part,  la  prise  de  decision  en  ce  qui  concerns  la  validation  ou  non  d'un 
recalage  est  au  moins  aussi  bonne  que  celle  d'un  navigateur; 

-  d'autre  part,  pour  le  diagnostic  des  sources  d'erreurs,  nous  avons  atteint  un  taux 
de  detection  des  anomalies  de  95%  et  un  taux  de  fausses  detections  voisin  de 
0%; 

-  enfin,  en  corotlaire  des  deux  points  precedents,  le  systems  expert  ayant  effectue 
des  choix  plus  judicieux  tant  pour  la  validation  des  recalages  que  pour  la 
selection  de  la  meilleure  navigation,  les  performances  de  navigation  obtenues 
sont  globalement  meilleures. 

Ce  dernier  r6sultat  augments  I'intAret  du  systems  expert  en  raison  de  I'amblioration  de  la 
securite  du  vol  (localisation  precise  en  phase  de  penetration)  et  de  I'efficacite  de  la 
mission  (precision  de  la  position  et  de  la  vitesse  pour  I’mitialisation  des  armes). 

Par  ailleurs,  cette  premiere  phase  a  permis  de  mettre  en  place  un  d6but  d'approche 
methodologique  pour  le  projet,  en  utilisant  une  methods  d6je  reconnue  et  en  la 
compietant  par  un  pnncipe  de  validation  specifique  au  projet. 


CONCLUSIONS  ET  PERSPECTIVES 

Comma  I'attestent  les  r6sultats  exposes  au  paragraphs  precedent,  la  maquette  de 
demonstration  realises  dans  le  cadre  de  cette  etude  a  r6ussi  A  montrer  la  validity, 
I'ir.teret  et  la  faisabilite  d'un  systems  expert  pour  la  surveillance  des  systAmes  inertiels 
et  I’aide  A  leur  mise  en  oeuvre. 

Cependant,  avant  d'aboutir  A  un  produit  opArationnel,  plusieurs  Atapes  semblent 
nAcessaires  • 

-  tout  d'abord  un  complement  de  la  base  de  connaissances  afin  de  prendre  en 
compte  un  nombre  plus  important  de  sources  d'erreurs, 

•  ensuite  la  conception  de  I'ergonomie  de  I’interface  d'un  tel  systAme  avec  le  pilots, 

-  et  enfin,  I'Atude  de  I'embarquabilitA  du  logiciel  ainsi  6crit  dans  un  environnement 
opArationnel. 

Enfin,  le  systAme  prAsentA  ici  s'intAgrera  tout  naturellemment  au  projet  de  copilots 
Alectronique. 
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ABSTRACT 

This  paper  describes  an  ongoing  program  which  is  developing  an  effective,  real-time,  knowledge -based 
decision-aiding  system  for  air  combat.  The  Integrated  Control  and  Avionics  for  Air  Superiority  (ICAAS) 
advanced  development  program  is  sponsored  by  the  Wright  Research  and  Development  Center  (WRDC)  at  Wright- 
Patterson  Air  Force  Base,  Ohio.  A  research  and  development  contract  was  awarded  to  the  McDonnell  Aircraft 
Company,  St  Louis,  Missouri,  in  September  1987.  The  program  objective  is  to  develop,  integrate,  and 
demonstrate  critical  technologies  which  will  enable  United  States  Air  Force  tactical  fighter  aircraft  to 
kill  and  survive  when  outnumbered  as  much  as  four  to  one  by  enemy  aircraft  during  air  combat  engagements. 
Primary  emphasis  is  placed  upon  beyond-visual- range  (BVR)  multiple  target  attack  capability  with  provisions 
for  effective  transition  to  close-in  combat.  Knowledge-based  pilot  decision-aiding  techniques  and  expert 
system  methods  are  used  to  achieve  substantially  enhanced  offensive  and  defensive  capabilities  compared  to 
current  operational  systems.  Situation  awareness  information  and  recommended  actions  are  computed  to  aid 
the  pilot  in  selecting  the  most  effective  attack  and  defend  engagement  options.  The  system  maximizes 
opportunities  for  missile  launch  against  multiple  enemy  aircraft  while  maintaining  options  to  defend  when 
necessary.  Sufficient  integration  and  automation  are  provided  for  application  to  a  single  seat  fighter 
An  intraflight  data  link  is  used  for  enhanced  mutual  support  between  individual  flight  members  to  make  them 
a  more  effective  combat  team. 


SYSTEM  INTRODUCTION 

Specific  functions  of  the  ICAAS  system  include  attack  management,  tactics,  attack  guidance,  defensive 
assets  manager,  and  aircraft  performance  monitor  as  illustrated  in  Figure  1.  Attack  management  automatically 
controls  onboard  sensors  and  computes  a  correlated  summary  of  track  files  from  ownship  sensors  as  well  as 
information  received  over  the  data  link  from  other  flight  members  The  track  file  summary  is  displayed  to 
the  pilot  to  maximize  his  situation  awareness.  The  tactics  function  uses  a  rule-based  approach  which  blends 
a  number  of  knowledge-based  values  to  assist  the  pilot  in  assessing  the  overall  situation.  Scenario 
attributes  are  evaluated  in  real-time  against  a  tactics  knowledge  base,  extracted  from  expert  pilots,  to 
recommend  the  most  viable  tactics.  Specific  coordinated  assignments  are  provided  to  each  flight  member 
Attack  guidance  provides  flight 
trajectory  information  for 
achieving  missile  launch  solutions 
against  multiple  enemy  aircraft 
while  maximizing  ownship  survival. 

Faster  than  real-time  missile  fly¬ 
out  models  are  used  to  perform 
engagement  predictions  and  guide 
the  aircraft  to  recommended  launch 
points.  The  defensive  assets 
manager  is  activated  by  the  sensed 
or  inferred  launch  of  a  threat 
missile  against  ownship.  An 
extensive  data  base  of  early, 
midcourse,  and  endgame  maneuvers  is 
used  to  generate  and  validate 
available  evasive  options  and  to 
select  the  most  promising  for 
recommendation  to  the  pilot.  The 
aircraft  performance  monitor  uses 
stored  knowledge  of  aircraft 
performance  parameters  to  aid  the 
pilot  in  achieving  the  best  real¬ 
time  dynamic  aircraft  performance. 

The  knowledge -based  decision-aiding 
features  of  these  specific  ICAAS 
system  functions  are  addressed  in 

more  detail  in  the  following  Figure  1.  ICAAS  System  Functions 

paragraphs . 


The  ICAAS  Attack  Management  (IAM)  function  provides  a  high  degree  of  situation  awareness  to  the  pilot. 
It  forms  an  interface  between  the  pilot  and  his  onboard  sensors  and  weapons.  IAM  manages  multiple  sensors 
and  correlates  track  files  from  ownship  sensors  as  well  as  other  flight  members  to  form  a  single  picture 
of  current  threat  location  and  identification.  Sensors  are  also  coordinated  to  acquire  sufficient  data  to 
launch  and  guide  weapons.  The  IAM  fire  control  function  identifies  current  launch  opportunities  and  provides 
aim-dot  steering  cues  plus  other  launch  parameters  against  the  primary  targets.  The  pilot  maintains  control 
of  search  volume ,  emission  mode,  and  launch  decisions.  The  IAM  function  is  a  special  adaptation  specifically 
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for  ICAAS  of  the  Air* to -Air  Attack  Management  (A3M)  system  developed  by  the  WRDC  Avionics  Laboratory. 
Reference  I  gives  a  detailed  description  of  A3M.  Individual  features  of  IAM  are  presented  below. 


Sensor  Control  -  Pilot  workload  associated  with  the  task  of  individual  sensor  control  can  be  very 
high.  A  modern  radar  has  more  than  twenty  different  operating  inodes,  and  a  pilot  can  easily  dedicate  too 
much  of  his  attention  to  operating  the  radar.  Since  future  fighters  will  likely  have  multiple  sensors, 
the  pilot  of  a  single  seat  fighter  can  be  overwhelmed  if  required  to  manually  operate  all  sensors. 

tam  s»nsor  control  logic  commands  each  of  the  ownship  sensors  as  required  to  update  all  track  files 
according  to  their  urgency  as  well  as  to  search  the  volume  defined  by  the  pilot  or  the  tactics  algorithm. 
The  knowledge-based  sensor  control  logic  accounts  for  the  availability  of  each  sensor,  its  particular 
operating  modes,  and  the  level  of  emissions  allowed  by  the  pilot.  The  emission  modes  provided  are.  active 
with  full  sensor  emissions,  low  probability  of  intercept  with  limited  emissions  to  periodically  acquire 
range  as  required,  and  passive  using  only  non-emitting  sensors  such  as  an  Infrared  Search  and  Track  System 
(IRSTS).  The  pilot  can  assume  total  control  of  any  individual  sensor  if  he  chooses. 

Multi-Source  Integration  -  In  current  fighter  aircraft,  information  from  individual  sensors  is 
presented  to  the  pilot  on  separate  cockpit  displays.  The  pilot  must  mentally  correlate  the  information 
into  a  common  frame  of  reference.  This  limits  his  ability  to  establish  and  maintain  a  good  awareness  of 
the  tactical  situation.  The  ICAAS  multi -source  integration  (MSI)  feature  correlates  sensor  track  files 
from  multiple  ownship  sensors  into  a  single  track  file  summary.  MSI  track  files  are  derived  from  kinematic 
and  covariance  data  reported  from  individual  sensors.  A  nearest  neighbor  approach  is  used  to  examine  sensor 
reports  for  correlation  with  existing  MSI  track  files  Track  files  are  then  created,  updated,  propagated, 
or  deleted  as  appropriate. 

Target  identification  is  derived  using  a  heuristic  voting  method.  Each  sensor  provides,  when 
available,  information  regarding  target  identification  (friend,  foe,  neutral),  class,  and  type.  Information 
from  each  sensor  is  weighted  and  combined  with  other  sources  to  produce  a  weighted  sura.  Comparison  with 
specified  thresholds  is  used  to  establish  a  confidence  factor. 

Internettine  -  An  intraflight  data  link  between  ICAAS  flight  members  permits  rapid  exchange  of  track 
files,  targeting  assignments,  ana  selected  tactics.  This  capabiliry  is  referred  to  as  internetting 
Additional  information  may  be  available  from  wide  area  data  network  sources  such  as  Airborne  Warning  and 
Control  System  (AWACS)  or  ground  control,  but  the  intraflight  information  exchange  is  the  focus  for  ICAAS 
application  (see  Figure  2) 

The  IAM  internetting  function  compiles  a  correlated  summary  of  track  files  which  are  received  from 
all  fighters  This  improves  threat  track  file  accuracy  and  increases  identification  confidence.  The  pilot 
of  each  aircraft  within  the  intraflight  network  can  view  all  track  files,  from  his  own  geometric  viewpoint, 
and  can  distinguish  between  internetred  and  ownship  MSI  track  files  IAM  internetting  dramatically  increases 
situation  awareness  of  the  target  environment  with  little  or  no  voice  communication  required  Sensor  scan 
volumes  can  be  coordinated  to  provide  maximum  total  coverage  and  thus  reduce  the  probability  of  undetected 
threats  Two  internetted  fighters  can  triangulate  with  line-of-sight  passive  sensors,  such  as  infrared, 
to  accurately  determine  target  range  while  both  remain  covert.  Accurate  relative  position  information  can 
also  reduce  the  need  to  maintain  visual  contact  between  flight  raembert 

Fire  Control  -  Baseline  weapons  for  ICAAS  fire  control  system  functions  are  the  advanced  medium  range 
air-to-air  missile  (AMRAAM) ,  AIM-9  sidewinder,  and  20mm  gun.  Data  from  MSI  and  internetted  track  files  pits 
embedded  missile  fly-out  models  are  used  to  compute  ownship  missile  launch  envelopes  and  threat  missile 
envelopes  against  the  ownship 

During  internetted  operation,  a  cooperative  attack  subfunction  coordinates  targets  with  each  blue 
fighter  within  the  flight  according  to  the  assignments  by  the  blue  pilot  tactical  lead  or  by  the  ICAAS 
Tactics  Algorithm.  Each  pilot  can  alter  his  own  attack  sequence  as  recommended  by  the  tactics  algorithm. 
Aim-dot  steering  is  provided  for  the  top  two  priority  targets,  and  the  sensor  manager  ensures  that  ownship 
sensors  are  tracking  these  targets. 

An  internetted  fire  control  capability  called  cooperative  launch  is  being  conceptually  developed 
This  allows  one  fighter  to  launch  an  AMRAAM  missile  which  will  be  guided  toward  the  target  by  another 
friendly  fighter.  The  launching  aircraft  can  approach  the  engagement  as  covertly  as  possible,  execute  the 
launch  based  on  target  track  data  supplied  by  the  other  flight  members  over  the  data  link,  then  immediately 
disengage  to  avoid  lethal  enemy  launch  envelopes.  From  a  greater  standoff  range,  the  guiding  aircraft  can 
provide  roidcourse  guidance  for  the  blue  missile.  Meanwhile,  the  launching  fighter  can  re-enter  the 
engagement  in  support 


TACTIC?  AWRITBM 

As  a  pilot  approaches  a  potential  BVR  engagement,  the  ICAAS  tactics  function  assists  with  selection 
and  implementation  of  an  appropriate  tactic  to  achieve  a  first  missile  launch  opportunity.  A  " tactic" 
consists  of  target  allocation  and  prioritization,  intercept  geometry,  and  a  strategy  for  electronic 
emissions. 

The  tactics  algorithm  consists  of  four  principal  elements:  menu  generator,  checkpoint  generator, 
monitor,  and  internetting  executive  The  menu  generator  makes  tactical  recommendations  based  on  real-time 
threat  information  from  IAM,  blue  aircraft  status,  and  a  tactics  knowledge  base  Checkpoint  generator 
defines  the  selected  tactic  in  terms  of  relative  geometry  goals,  airspeed,  aircraft  radar  or  infrared 
signature  management,  and  electronic  emissions  strategy.  The  tactic  monitor  modifies  checkpoints  as  the 
engagement  unfolds  and  evaluates  the  success  of  the  tactic  in  terms  of  accomplishing  desired  goals.  The 
internetting  executive  coordinates  tactics  algorithms  which  are  executing  Independently  on  internetted 
aircraft.  The  pilot  can  tailor  the  response  of  the  tactics  algorithm  according  to  personal  preferences  or 
new  threat  tactics  experienced  in  earlier  engagements  by  altering  preflight  input  parameters  which  affect 
tactics  calculations.  Examples  include  preferred  tactic  categories  and  desired  launch  ranges. 


Tactics  options  are  determined  by 
assessing  cite  enemy  (red)  formation  and  friendly 
(blue)  formation  capabilities.  Candidate 
tactics  are  then  generated  and  rank  ordered 
within  the  context  of  the  engagement  scenario. 

This  process  is  illustrated  in  Figure  3. 

The  tactics  knowledge  base  designed  for 
ICAAS  is  applicable  to  the  following  three 
scenarios  for  bcth  the  F-1S  and  a  mid*90's 
fighter: 

-  AWACS  defense  against  a  high,  fast 
threat 

•  Fighter  escort  of  bombers 

-  Base  defense 

Nine  pilots  were  interviewed  to  determine 
current  day  tactical  objectives  for  each  of  the 
ICAAS  scenarios  The  pilots  interviewed 

represented  a  combined  total  of  over  17,000 
flight  hours  in  both  Tactical  Air  Command  (TAC) 
and  Air  National  Guard  (ANG)  units.  Over  half  Figure  2.  Data  Link  Network 

of  these  hours  wore  in  the  F-15.  Interviews 

were  structured  to  solicit  information  regarding  geometry,  speeds,  altitudes,  sensor  management,  target 
sorting  and  prioritization,  missile  selection  and  launch  range,  and  salvo  strategy. 

Digital  off-line  analyses  and  rapid  prototyping  were  conducted  to  refine  the  tactic  specifications 
and  tolerances.  Scenarios  were  executed  from  early  set-up  to  first  launch  position  Shot  opportunities 
and  steering  philosophies  were  then  evaluated.  The  result  is  a  set  of  35  different  tactics  which  are 
characterized  by.  mission  type,  intercept  goals,  altitude  relative  to  the  threats,  speed  of  blue  aircraft, 
relative  position  of  the  blue  aircraft,  and  sensor  strategy.  Each  tactic  is  defined  for  both  the  tactical 
lead  and  the  wingraan  aircraft.  The  initial  knowledge  base  contains  current  operational  tactics  which  are 
tuned  for  the  ICAAS  weapons  and  avionics  suites.  Work  is  continuing  in  conjunction  with  Tactical  Air  Command 
pilots  to  expand  the  knowledge  base  with  more  innovative  tactics  designed  to  exploit  the  intraflight  data 
link  and  signature  features  of  an  advanced  aircraft. 


Menu  Generator  ■  A  tactics  option  menu  is  computed  using  the  logic  tree  structure  depicted  in  Figure 
4.  The  most  fundamental  de'.erminant  in  tactic  selection  is  blue  mission  as  the  first  node  of  the  tree 
Branches  emanating  from  this  node  represent  the  three  mission  scenarios  which  are  implemented  in  the  tactics 
algorithm.  Secondary  nodes  of  the  tree  assess  the  nature  of  the  threat  within  geometric  and  kinematic 
constraints  of  the  appropriate  mission.  Processing  the  relative  geometries  and  velocities  determines  a  level 
of  threat  along  each  red  aircraft 
velocity  vector.  Branching  occurs 
according  to  urgency  in  dealing  with 
the  threat.  Subsequent  nodes  in  the 
tree  are  activated  by  situational 
parameters  such  as  altitude  of  Che 
throats ,  number  and  type  of  blue 
defenders,  and  the  rules  of  engagement 
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At  the  end  of  any  set  of  tree 
branches  is  the  list  of  applicable 
tactics  for  that  particular  node  and  a 
stored  base  figure  of  merit  (FOM)  for 
each.  The  tactics  Included  in  each 
subset  and  corresponding  FOMs  are  the 
result  of  pilot  interviews  and  off-line 
analyses  using  digital  air  battle 
simulations  The  aircraft  performance 
emphasis  within  these  different  tactics 
also  varies,  from  a  full  afterburner 
direct  approach  to  a  minimum  fuel 
consumption  loiter. 

Target  assignments  and  launch 
sequences  are  computed  for  a  particular 
tactic,  subject  to  priority  inputs  from 
the  pilot  The  solution  is  a  function 
of  the  blue  mission,  threat  positions 
relative  to  a  high  value  asset,  and 
target  tactical  priority  Sorting 

between  two  blue  aircraft  considers  threat  separation  in  azimuth  and  elevation  so  a  single  blue  is  not 
responsible  for  widely  separated  threats.  Launch  sequencing  then  results  from  the  intercept  geometry  of 
the  tactic  along  the  attack  axis.  The  pilot's  assignments  and  sequences  are  considered  top  priority  for 

tmi.il  blue  aiitLat Targets  assigned  by  the  algoiithm  al'e  adjusted  if  a  blue  aircraft  haS  Insufficient 

mirsiles  to  attack  all  assigned  targets. 
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Figure  3.  Tactic  Options  Menu  Generation 


Relative  ranking  of  tactics  options  is  determined  using  the  final  subset  of  tactics  defined  by  the 
logic  tree.  The  base  FOM  stored  with  each  tactic  is  incremented  or  decremented  according  to  weighting 
factors  assigned  to  the  sixteen  engagement  attributes  shown  in  Figure  5.  Final  values  are  normalized  so 
the  FOM  assigned  to  each  tactic  has  a  value  between  0  (very  poor  tactic)  and  1.0  (very  good  tactic)  Tactics 


are  ranked  in  descending  order  of  FOM  end  made  available  for  cockpit  display.  The  lead  pilot  is  free  to 
select  any  tactic  from  the  available  menu. 


8asa 


BVR  -  Beyond  Visual  Range 
DR  -  Direct  Route 
HFF  **  High  Fast  Flyer 
IP  -  Interpose 
LO  -  Low  Observables 
ROE  -  Rules  of  Engagement 
WVR  -  Within  Visual  Range 


Figure  4.  Partial  View  of  Tactics  Geometry  Tree 


Checkpoint  Generator  -  Once  the  flight  leader  has  selected  a  tactic,  a  series  of  checkpoints  is 
computed.  These  checkpoints  provide  important  information  for  effective,  coordinated  execution  of  the  tactic 
including  flight  path  constraints,  goals  for  energy  management,  sensor  emission  levels  and  scan  volumes, 
target  priorities,  countermeasure  employment  cues,  and  first  weapon  employment  cues.  Wingmen  are 
automatically  provided  their  respective  checkpoints  within  the  coordinated  tactic.  Thus,  each  flight  member 
knows  his  own  individual  assignments  and  the  flight  can  operate  as  an  effective  team.  Figure  6  illustrates 
initial  checkpoint  generation  for  two  aircraft  executing  a  single* side  offset  tactic. 


Associated  with  a  set  of 
checkpoints  is  a  set  of  time  or  spatial 
tolerances.  These  are  the  blue  flight 
path  tolerances,  the  checkpoint  tactical 
tolerances,  and  the  red  maneuver 
tolerances.  The  blue  flight  path 
tolerances  are  Che  most  restrictive 
because  the  attack  guidance  function 
performs  path  predictions  and 
optimization  within  these  tolerances. 
Checkpoint  tactical  tolerances  allow  for 
variation  in  the  implementation  of  a 
tactic  by  the  pilot  without  readjustment 
of  the  checkpoint  parameters  and  the 
ensuing  shifts  in  the  predicted  path. 
The  red  maneuver  tolerance  defines  a  set 
of  threat  reactions  within  which  a 
particular  tactic  automatically  makes 
adjustments.  Outside  of  this  tolerance, 
the  tactic  FOM  will  be  adjusted  to 
reflect  a  situation  which  is  not 
progressing  as  well  as  planned 

Tactics  Monitor  -  A  tactics 
monitor  subfunction  continually  monitors 
progress  of  a  selected  tactic  to  see  if 
the  desired  results  are  being  achieved. 
A  general  monitor  observes  those 
parameters  of  air-to-air  engagements 
which  are  similar  regardless  of  the 
specific  tactic  being  implemented. 
Examples  are  a  change  in  target 
assignments  or  threat  formation,  or  the 
sudden  detection  of  a  new  threat  at  or 
tiaat*  a  Iminoh  qnluHfln  on  blue  The 
scenario  events  checked  by  the  ICAAS 
general  tactic  monitor  are  listed  in 
Figure  7  along  with  the  monitor 
response.  The  general  monitor  also 
adjusts  the  FOM  for  the  selected  tactic 
based  on  changing  conditions  as  the 
scenario  unfolds. 
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A  tactic- specific  monitor  observes 
those  characteristics  which  are  peculiar 
to  the  currently  selected  tactic,  updates 
the  parameters  of  the  checkpoints,  and 
determines  the  sequencing  to  the  next  step 
in  the  tactic  plan.  For  example,  the 
tactic- specific  monitor  recomputes  the 
amount  of  time  needed  Co  obtain  a  desired 
offset  and  adjusts  the  guidance 
specification  and  tolerances  for  attack 
guidance  accordingly.  When  the  offset  is 
achieved,  the  tactic-specific  monitor 
triggers  the  next  action,  for  example, 
recommended  activation  of  a  radar  or  ECM. 

The  tactic-specific  monitor  also  reacts  to 
a  degraded  tactical  situation.  For 
example,  if  the  blue  pilot  flies  outside 
of  the  current  checkpoint  tolerance, 
alternative  tactics  are  evaluated. 

The  tactic-specific  monitor  also 
contains  a  knowledge  base  for  the  expected 
set  of  threat  responses  for  each  tactic. 

Such  parameters  as  spatial  locations  of 
checkpoints,  offset  angles,  or  target 
Intercept  angles  are  altered  if  the  threat 
response  fall*  within  the  threat  maneuver 
tolerance.  Figure  C  illustrates  a  modification  to  the  initial  single  side  offset  tactic  due  to  an  altered, 
but  anticipated,  threat  response.  The  FOM  is  significantly  decremented  if  the  threat  package  response  falls 
outside  tho  defined  threat  maneuver  tolerance. 
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Figure  6.  Tactic*  Initial  Checkpoints 


Figure  7.  Tactics  Algorithm  General  Monitor 


The  tactics  monitor  updates  the 
FOM  for  the  selected  tactic  as  the 
engagement  unfolds.  It  compares  the 
current  FOM  to  a  predefined  threshold 
input  by  the  pilot.  This  minimum  FOM  is 
used  to  characterize  a  desperate 
situation  when  a  tactical  disengage 
should  be  recommended.  If  the  FOM  ofc 
the  current  tactic  is  decreasing,  a 
search  for  a  potentially  better  tactic 
is  Initiated.  A  recommendation  to 
switch  to  a  better  tactic  Is  made  if  a 
significant  improvement  is  available. 
In  all  cases,  the  tactics  algorithm 
provides  only  an  advisory  to  the  pilot. 
He  still  makes  the  decision  to  continue 
with  the  existing  tactic  or  to  select  an 
alternative. 

Xn£-g,i:n^^lDfc-^P,Villy.g  -  Tactics 
algorithms  in  each  blue  aircraft  operate 
independently  of  each  other.  To 
coordinate  the  menu  generation  and 
tactics  monitoring  processes  between  the 
blue  lead  and  his  wlngman,  key  tactics 
algorithm  data  and  sensor  information 
from  each  blue  aircraft  are  exchanged 
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over  the  intraflight  data  link.  The  executive  nodule  (exec)  then  coordinates  thr  Interaction  between  the 
tactics  algorithms  of  each  aircraft.  Whe:.  two  or  more  blue  aircraft  are  present  and  the  Internetting  data 
link  is  valid,  exec  checks  the  role  of  Uu«  ownship  (lead  or  wing).  In  the  lead  aircraft,  the  exec  computes 
the  group  FOM  (average  of  lead  and  wing  individual  FOHs)  which  is  displayed  to  both  blue  pilots.  In  the 
wingman  aircraft,  the  exec  ensures  that  the  wingaan  is  viewing  the  same  tactic  as  the  lead  pilot.  Thus, 
the  wingman' s  display  remains  in  unison  with  the  flight  leader.  Also,  the  exec  ensures  that  only  the  lead 
pilot  can  select  a  tactic  for  implementation  by  both  blues. 

When  the  internetting  data  link  is  not  valid,  the  action  of  the  tactics  algorithm  depends  upon  whether 
a  tactic  had  been  selected  prior  to  the  net  going  down.  If  no  tactic  was  selected,  the  lead  and  wing  pilots 
coordinate  target  assignment  and  tactic  selection  by  voice  call,  each  using  his  tactics  algorithm  with 
situational  data  provided  by  ownship  sensors.  If  a  tactic  was  being  implemented  when  the  data  ) ink  wont 
down,  exec  determines  if  the  remaining  ownship  checkpoints  of  the  tactic  can  be  continued.  This  is  a 
function  of  the  continued  availability  of  track  files  on  the  assigned  targets  and  the  steering  philosophies 
associated  with  the  remaining  checkpoint  legs.  If  the  tactic  can  be  continued,  the  tactic  monitoring 
function  in  each  blue  aircraft  continues  to  process  its  respective  set  of  checkpoints.  If  the  tactic  cannot 
be  continued,  exec  initiates  a  new  menu  generation  based  on  non-internetted  operation. 

AHACK  GUIDANCE 

Primary  functions  of  attack  guidance  are  to  generate  flight  path  trajectories  which  will  satisfy 
criteria  associated  with  the  desired  tactic  and  to  compute  recommended  launch  points  against  multiple 
assigned  targets.  It  is  designed  to  enhance  pilot  awareness  of  the  engagement  by  providing  Information  on 
predicted  flight  paths  and  missile  impact  zones.  Specific  kill  and  survival  metrics  associated  with  each 
assigned  target  indicate  the  likelihood  of  target  kill  as  well  as  danger  to  the  blue  aircraft  from  all 
thieat  aircraft.  Attack  guidance  overcomes  a  limitation  in  current  fighter  attack  systems  where  only  the 
highest  priority  target  is  designated  for  attack,  and  a  fire  control  solution  Is  computed  without 
consideration  of  other  threats  to  ownship.  ICAAS  attack  guidance  considers  the  overall  situation  and  future 
ownship  and  threat  position,  thus  greatly  increasing  pilot  awareness  of  the  future  consequences  of  his  near 
term  actions. 


The  heart  of  attack  guidance  is  an  engagement  predictor.  This  predictor  uses  embedded  knowledge  of 
ownship  oerformance  capabilities,  energy  management  schedules,  tactical  goals,  plus  red  and  blue  weapon 
envelopes  to  predict  future  relative  positions  and  progress  of  the  engagement.  A  flight  plan  Is  initially 
constructed  based  on  Inputs  from  the  tactics  algorithm  and  the  pilot,  such  as  target  assignments,  tactics 
checkpoints,  and  ownship  aircraft  speed  and  altitude  goals  A  four •dimensional  (time  and  space)  trajectory 
Is  generated  which  is  responsive  to  the  goals  and  constraints  from  the  flight  plan  Recommended  launch 
points  are  determined  for  each  individual  target  in  conjunction  with  kill  and  survivability  metrics. 
Predictions  are  continuously  updated  during  all  mission  phases  to  account  for  both  ownship  and  threat 
aircraft  maneuvering.  Outputs  are  used  to  drive  cockpit  displays  and  to  generate  steering  commands  for  pilot 
manual  control.  An  automatic  flight  control  coupler  Is  also  available  If  activated  by  the  pilot.  Figure  9 
illustrates  the  features  of  attack  guidance  versus  two  red  aircraft. 

The  trajectory  predictor 


algorithm  Integrates  differential 
equations,  which  describe  the 
aircraft  motion,  while  controlling 
the  aircraft  through  a  set  of 
autopilot* like  guidance  laws.  The 
Integration  process  takes  place  at 
a  rate  many  times  faster  than  real 
time.  Variable  Integration  step 
sizes  are  used  so  that  near  term 
predictions  are  computed  more 
accurately  than  those  further  In 
the  future,  thus  reducing  the 
computer  throughput  requirements . 
The  fast  fly-out  computations 
consist  of  five  parts;. 

Extrapolate  enemy 
aircraft  position  based 
upon  altitude  and 
airspeed  at  the  time 
prediction  update 
begins 

Computate  error 
signals  at  every 
integration  step  to 
drive  simulated  blue 
aircraft  into  capturing 
the  desired  aircraft 
state  goals  contained 
in  the  flight  plan;- 
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-  Generate  4D  Trajectories 
Update  Predictions 

Support  Engagement  Execution 
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Figure  9.  Attack  Guidance 


-  Integrate  a  set  of  differential  equations  describing  aircraft  motion; 

-  Determine  control  adjustments  required  to  achieve  the  profile  constraints  specified  in  the 
flight  plan;  and 

-  Curve  fit  the  initial  portion  of  the  predicted  profile  to  generate  Input  data  for  the  Control 
Coupler . 
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Flight  path  predictions  and  associated  flight  guidance  algorithms  are  tailored  to  support  four  mission 
phases:  enroute,  tactic,,  attack,  and  disengage. 

-  Enroute  Phase:*  Enroute  trajectory  planning  is  provided  for  those  portions  of  flight  just 
before  and  after  the  threat  engagement.  Trajectory  predictions  are  based  on  embedded 
performance  schedules  for  optimal  climb,  cruise  and  descer t  speed,  and  altitude.  These 
schedules  account  for  aircraft  weight,  stores,  and  actual  unt lent  conditions. 

It  is  as;umed  that  all  navigation  waypoints  have  an 
assigned  altitude.  Speed  may  be  fixed  or  free.  If 
speed  is  ijee,  an  optimal  value  is  selected,  either 
maximum  range  for  waypoint  steering  or  maximum 
endurance  for  CAP  orbits.  Direct  path  steering  is 
used  to  navigate  from  waypoint  to  waypoint,  flying 
over  the  waypoint  on  the  inbound  leg.  The  logic 
includes  the  ability  to  capture  and  hold  a  CAP  orbit 
with  a  pilot-specified  leg  length  and  heading.  CAP 
orbit  is  maintained  until  an  exit  command  is  given  by 
the  pilot. 

-  Tactic  Phase:  The  checkpoints  defined  by  the 
tactics  algorithm  are  typically  stated  in  terms  of  a 
desired  geometry  relative  to  the  threat  aircraft, 
and,  as  such,  do  not  constitute  an  inertial  flight 
path.  Engagement  predictions  applies  the  guidance 
laws  prescribed  by  tactics  (such  as  line-of-sight 
offset)  against  an  extrapolated  threat  flight  pa'h  to 
evolve  a  four-dimensional  inertial  trajectory 
suitable  for  cockpit  display. 

-  Attack  Phase'  This  processing  computes  the  attar l  trajectory  against  each  assigned  target 
aircraft.  Embedded  in  the  generation  of  the  trajectory  are  the  kill  and  survival  metrics  and 
the  recommended  launch  points  against  the  assigned  targets.  The  attack  path  starts  when  the 
tactic  algorithm  first  launch  objective  has  been  achieved  or  when  the  pilot  selects  pure  attack 
steering.  The  attack  trajectory  ends  following  the  predicted  (or  actual)  launch  against  the 
last  assigned  threat  aircraft  and  completion  of  target  illumination.  Attack  processing  supports 
multiple  launches  against  a  single  target  as  wall  as  multiple  missiles  inflight  against  multiple 
targets . 

The  attack  trajectory  is  constructed  sequentially  from  target  to  target  based  on  the  defined 
launch  sequence.  Blue  launch  criterion  is  based  on  achieving  a  desired  launch  range  within  a 
defined  level  of  survivability.  Both  the  launch  range  goals  and  the  minimum  acceptable 
survivability  are  input  by  the  pilot.  The  6  sired  launch  range  is  measured  in  percent  of 
maximum  missile  launch  range  (RMAX) .  Survlvi.MUty  is  a  function  of:  the  total  number  of 
threat  aircraft,  the  danger  each  threat  lipooaa  based  on  maximum  missile  ranges,  and  the 
predicted  success  of  blue  missiles.  Post  1«  uwh  missile  illumination  requirements  are  Imposed 
on  the  trajectory. 

•  Disengage  Phase:  The  disengage  trajectory  lS  a  maneuvering  flight  path  to  escape  danger  from 
all  threat  aircraft,  followed  by  a  direct- v  >  navigation  path  to  a  designated  regroup  point. 
Determination  of  the  danger  posed  by  the  combat  situation  during  disengage  processing  is 
Illustrated  in  Figure  10.  Ownship  is  declared  to  be  clear  of  the  engagement  if:  all  threat 
aircraft  are  behind  a  line  through  the  ownship  position  perpendicular  to  the  line-of-sight  from 
ownship  to  the  regroup  point,  all  threats  are  farther  away  than  a  pre- defined  distance,  and 
ownship  heading  is  within  90  degrees  of  the  bearing  to  the  regroup  point. 

pEFENSIVF.  ASSETS  MANAGER 

The  Defensive  Assets  Manager  (DAM)  responds  to  detection  of  air-launched  threat  missiles  and  provides 
the  fighter  pilot  with  a  recommended  defensive  response  derived  from  a  stored  knowledge  base  Options 
considered  for  response  include  infrared  and  radar  countermeasures  (expendable  and  non- expendable)  applied 
in  conjunction  with  early,  midcourse,  and  endgame  aircraft 
maneuvers.  Early  kinematic  maneuvers  to  either  avoid  or 
penetrate  and  escape  the  maximum  lethal  intercept  zone  are 
highly  effective  when  the  threat  missile  has  been  launched 
from  long  range.  If  the  threat  missile  was  launched  in  a 
no  escape  condition,  midcourse  maneuvers  with 
countermeasures  are  used  to  defeat  the  missile  seeker, 
followed  by  endgame  evasive  maneuvers  if  required.  DAM 
options  are  illustrated  in  Figure  11. 

DAM  is  initiated  when  onboard  sensors  detect  a 
missile  launch, or  missile  in  flight.  Missile  launch  can 
also  be  inferred  from  threat  aircraft  actions.  Multiple 
missiles  t** y  be  reported  at  any  given  time,  but  only  the 
single  highest  priority  threat  is  reported  to  the  evasion 
algorithm  for  defensive  response.  A  DAM  scheduler 
eliminates  missiles  which  are  not  a  threat  to  ownship 
through  range,  closure,  and  guidance  checks.  Remaining 
missile  reports  are  then  prioritized  according  to  predicted 


Figure  11.  Defensive  Assets  Manager 


Figure  10.  Danger  Zone  Clearance  for  Disengage 
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time  to  closest  approach. 

This  assures  that  the  missile 
selected  as  the  greatest 
threat  is  within  lethal 
range,  is  guiding  on  ownship, 
and  has  the  shortest  time-of- 
arrival . 

An  interface  between 
DAM  and  the  IAM  sensor 
manager  provides  special 

management  of  missile  track 
files.  IAM  initiates  and 

maintains  an  MSI  track 
following  a  launch  alert  from 
ownship  radar  or  IRSTS 

sensors.  Just  as  it  does  for 
any  new  aircraft  track 
acquisition.  Missile  tracks 
which  are  positively 

identified  as  a  threat  to 
ownship,  or  whose  threat 
status  is  unknown,  are  given 
top  track  service  priority 
over  all  other  tracks.  The 
predicted  near-term  missile 
flight  path,  computed  by  DAM,  '* 
can  be  used  to  help  re¬ 
acquire  a  missile  should  IAM 
fail  to  update  the  missile 
track  DAM  then  issues 
special  requests  on  the  Figure  12.  Defensive  Asset  Manager  Missile  Evasion  Algorithm 

missile  to  IAM  to  ensure 

active  MSI  track  files  on  known  threat  missiles  are  updated.  It  also  monitors  tracks  of  threat  aircraft 
with  a  high  probability  of  launch  as  determined  by  RWR  detection 

DAM  assembles  the  missile  track  reports  from  IAM  track  files  for  communication  to  the  scheduler  Due 
to  the  time  criticality  of  information  generated  by  the  Missile  Warning  Sensor  (MWS),  MWS  reports  are  not 
processed  by  IAM.  This  data  is  passed  directly  to  the  DAM  scheduler  for  endgame  response  recommendation 

The  missile  evasion  algorithm  is  activated  when  the  DAM  scheduler  determines  the  greatest  threat  to 
ownship.  The  evasion  algorithm  executes  once  per  second  as  long  as  a  missile  poses  a  threat  to  ownship 
The  major  functional  components  are  shown  in  Figure  12  and  are  described  below. 


-  To  determine  the  level  of  danger  to  ownship,  the  threat  missile  flight  path  is 
projected  forward  in  time  to  its  terminal  phase  through  the  use  of  an  embedded  high  speed  flyout  simulation. 
Ownship  is  assumed  to  fly  along  the  predicted  attack  path.  If  the  evasion  algorithm  determines  the  missile 
is  not  a  threat  to  ownship,  the  DAM  scheduler  records  this  particular  missile  as  no  threat  and  sends  the 
next  candidate  threat  missile  from  its  ordered  list  to  the  evasion  algorithm. 

Threat  assessment  must  be  capable  of  performing  when  incomplete  or  imperfect  missile  information  is 
available.  It  can  operate  using  sensor  data  repo-*-*'*  or.  the  missile  during  its  trajectory  or  from  the 
flyout  model  initiated  from  position  and  velocity  •  threat  aircraft  at  launch.  The  source  with  the 
least  uncertainty  is  used.  When  sensor  data  is  useu,  vyout  model  is  synchronized  to  help  smooth  data 
and  extrapolate  between  infrequent  or  noisy  data  poim  The  reported  missile  position  and  velocity  are 
adjusted  according  to  uncertainties  to  provide  a  "worst  case"  state  of  the  missile.  Uncertainty  associated 
with  sensor  data  will  typically  decrease  with  time.  However,  as  data  from  the  flycut  model  is  continuously 
used,  uncertainty  will  increase. 

Threat  assessment  also  attempts  to  establish  positive  identification  of  the  missile  type  If 
identification  cannot  be  established,  available  data  will  be  used  to  characterize  the  missile  as  much  as 
possible,  such  as  radar  guided  or  infrared  guided,  and  long  range  versus  short  range.  This  information  is 
needed  by  the  evasion  algorithm  to  evaluate  response  options. 

Qp-tlon  Generation  -  With  a  missile  determined  as  a  potential  threat,  a  list  of  candidate  evasion 
options  is  generated.  An  embedded  aircraft  performance  model  is  used  to  ensure  that  candidate  maneuvers 
are  achievable.  Early  and  raidcourse  options  are  constructed  to  explore  the  effectiveness  of  different 
combinations  of  heading  change  and  load  factor.  A  stored  knowledge  data  base  is  used  to  define  endgame 
evasive  maneuvers,  reactions  to  be  achieved  with  countermeasures,  and  conditions  under  which  each  option 
is  most  effective. 

Early  kinematic  maneuvers  are  designed  to  defeat  the  threat  missile  by  avoiding  the  intercept  zone, 
escaping  the  intercept  zone  t-he  missile  reaches  the  zone  boundary,  creating  a  missile  sensor  gimbal 
failure,  or  turning  to  generate  a  range  rate  failure.  A  constant  altitude,  maximum  sustained  turn  by  ownship 
is  assumed.  The  evasion  algorithm  identifies  options  to  provide  the  maximum  time  ownship  can  continue  on 
its  current  flight  path,  the  minimum  amount  of  heading  change  required,  and  the  time  ownship  must  follow 
a  radial  from  the  launch  site  to  escape  the  missile.  Gimbal  failures  are  created  by  taking  advantage  of 
the  missile  requirement  to  provide  a  lead  angle  for  intercept  to  occur.  Range  rate  failures  are  created 
by  taking  advantage  of  missile  guidance  system  sensitivity  to  small  closure  rates. 
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Midcourse  options  combine  countermeasures  with  maneuvers  to  defeat  the  threat  missile  seeker.  Constant 
altitude  turns  are  asauraed  at  or  below  the  maximum  sustainable  turn  load  factor.  Countermeasures  include 
flares  and  onboard  and  offboard  jammers. 

A  knowledge  base  of  candidate  endgame  options  was  identified  through  off-line  analysis  using  aircraft 
and  high  fidelity  missile  model  simulations.  A  barrel  roll  maneuver  proved  to  be  the  most  effective  against 
various  threat  missiles  because  it  is  least  sensitive  to  timing  and  produced  consistent  results.  The 
effectiveness  of  the  barrel  roll  is  stored  in  tabular  form. 

Potion  Ranking  -  Desirability  and  effectiveness  metrics  are  generated  for  use  in  ranking  the  candidate 
set  of  evasion  options.  The  desirability  metric  is  a  function  of  the  required  heading  change,  fuel  usage, 
time  required  to  complete  the  option,  and  mission  impact.  Mission  Impact  accounts  for  the  inability  to 
complete  illumination  for  an  AMKAAM  already  inflight.  An  option  is  considered  even  less  desirable  if  it 
forces  total  abandonment  of  an  offensive  posture  or  mission  objective.  The  effectiveness  metric  is  a 
function  of  the  ability  to  defeat  the  threat  and  the  uncertainty  associated  with  the  missile  track  files. 
Options  which  cause  hard  failures  of  the  missile  system  such  as  gimbal  failures,  closure  rate  limits,  or 
time* of -flight  limits  are  considered  more  effective  than  options  which  require  precise  timing  and  typically 
result  in  less  miss  distance,  such  as  endgame  maneuvers.  The  effectiveness  metric  accounts  for  the 
dependence  on  data  accuracy.  For  example,  the  desirability  of  an  option  that  is  critically  dependent  on 
accurate  missile  velocity  data  is  downgraded  over  one  that  requires  less  velocity  accuracy.  The  preferences 
of  the  pilots  are  also  reflected  in  the  effectiveness  metric.  The  products  of  the  desirability  and  the 
effectiveness  metrics  are  used  to  rank  order  the  candidate  options. 

The  top-ranked  option  is  validated,  before  it  is  recommended  to  the  pilot,  using  a  high  speed  missile 
flyout  model.  The  model  is  used  to  assess  effects  of  the  recommended  flight  path  and  countermeasure 
deployment  throughout  the  predicted  missile  time-of- flight .  An  option  is  considered  successful  if  the 
flyout  model  predicts  an  adequate  miss  distance. 

aircraet  msmmumm 

Pilots  of  today's  fighter  aircraft  do  not  get  much  assistance  from  onboard  systems  in  optimizing  the 
maneuver  potential  of  their  aircraft.  A  performance  manual  is  available  which  a  pilot  reviews  during  ground 
training,  but  he  is  forced  to  work  from  memory  when  airborne.  The  1CAAS  Aircraft  Performance  Monitor  (APM) 
uses  knowledge  of  ownship  aircraft  parameters  to  aid  the  pilot  in  achieving  the  best  real-time  dynamic 
performance  of  his  aircraft  in  an  effort  to  gain  a  performance  edge  over  his  adversary. 

APM  provides  real-time  corner  speed  and  angle-of-attack  (AOA)  cues  for  cockpit  display  The  angles- 
of -attack  correspond  to  best  acceleration,  best  sustainable  turn  rate,  or  maximum  Instantaneous  turn  rate, 
all  at  a  given  airspeed  and  altitude.  The  choice  of  the  particular  AOA  parameters  was  based  on  discussions 
with  pilots  to  determine  cues  that  would  help  make  optimal  use  of  aircraft  performance  during  an  air-to- 
air  engagement.  Only  one  AOA  c  is  displayed  at  any  given  time  based  on  pilot  intent  inferred  from  current 
stick  commands. 

The  AOA  cue  for  best  acceleration  was 
suggested  due  to  a  peculiar  characteristic  of  some 
aircraft  that  best  acceleration  is  achieved  under  a 
slightly  loaded  condition  in  some  regions  of  the 
flight  envelope.  The  pilots  familiar  with  this 
characteristic  reported  using  it  to  advantage  during 
combat  exercises.  The  cue  for  best  sustained  turn 
rate  at  current  airspeed  provides  a  zero  specific 
excess  power  (Ps)  Indicator  to  help  maintain  energy 
relative  to  an  opponent.  The  cue  for  maximum 
Instantaneous  turn  rate  was  suggested  to  effectively 
fly  near  the  structural  limit  without  violating  the 
structural  limit  boundary.  Current  overload  warning 
systems  advise  the  pilot  with  an  aural  tone  when  he 
is  near  the  limit,  but  pilots  still  fly  through  and 
beyond  the  limit  due  to  rapid  onset  rates.  A  visual 
cue  related  to  AOA  is  believed  to  be  more  effective. 

Pilots  typically  do  not  wish  to  fly  slower  than 
corner  speed  in  combat,  but  without  a  cue  they  use 
a  constant  value  rule-of-thumb  for  all  flight 
conditions.  A  cue  better  defines  the  corner  speed 
for  actual  flight  conditions  and  stores 
configuration.  These  performance  parameters  are 
pictorially  represented  on  the  maneuverability 
envelope,  or  "dog  house"  plot  shown  in  Figure  13. 

gXSim., 

The  Department  of  Defense  standard  Ada  software  language  is  being  used  for  1CMS  Implementation.  Most 
of  the  software  code  has  been  developed,  including  100,000  lines  or  850,000  32 -bit  words  of  executable  code 
plus  225,000  words  of  data  base  (Reference  2).  Rapid  update  rates  which  are  necessary  to  support  real-time 
iilasion  performance  led  to  a  computer  throughput  requirement  of  approximately  15  million  instructions  per 
second.  No  available  flightworthy  computer  with  this  capability  could  be  found,  so  a  new  computer  was 
developed  based  on  the  MIPS  Corporation  R-3000  32-bit  reduced  instruction  set  computer  (RISC)  processor. 
This  is  one  of  two  standard  processors  selected  by  the  Joint  Integrated  Avionics  Working  Group.  Each 
processor  is  conservatively  rated  at  five  million  instructions  per  second.  Three  processors  are  included 
in  a  single  computer  chassis,  resulting  in  a  throughput  capacity  of  15  million  instructions  per  second 
(Reference  3).  A  100  percent  reserve  wAs  achieved  by  using  two  of  these  computers  to  host  the  ICAAS 
software.  One  will  host  the  Tactics,  Attack  Guidance,  DAM,  and  APM  functions  and  is  designated  the  ICAAS 
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Figure  13.  Typical  Maneuverability  Envelope 
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Integration  Computer  (IIC).  the  second  computer,  the  ICAAS  Support  Computer  (ISC),  will  host  the  attack 
management  function.  These  computers  are  flightworthy  and  will  be  used  in  simulation  evaluations  as  well 
as  flight  test. 

Pilow/'^hicle  interface  (PVI)  is  a  major  ICAAS  system  integration  issue.  No  matter  how  much 
information  can  be  developed  vMf-Mn  the  system,  effective  ''"'"bet  performance  can  be  achieved  only  if  the 
pilot  can  easily  understand  and  apply  the  information.  Too  much  data  and  information  can  easily  overwhelm 
the  pilot.  Efficient  cockpit  data  management  techniques  are  needed,  including  a  proper  balance  between 
automation  and  manual  task  allocation. 

An  advanced  cockpit  instrument  panel  has 
been  designed  for  ICAAS  as  shown  in  Figure  14. 

Three  multifunction  color  displays  will  be 
used  to  present  tactical  situation  data, 
weapon  delivery  information,  threat  varnings, 
recommended  tactics,  and  flight  trajectories. 

A  large  center  display  provides  9  1/2  inches 
by  9  1/2  inches  of  viewing  area  as  the  primary 
situation  awareness  display.  The  other  two 
displays  are  6  Inches  by  6  inches.  A  helmet 
mounted  display  (HMD)  is  provided  ior  sensor 
cueing  and  threat  warning.  The  HMD  will  also 
be  used  to  cue  the  pilot  on  where  to  look  for 
targets  as  he  transitions  from  BVR  to  WVR, 
thus  providing  maximum  visual  acquisition 
range.  A  data  transfer  module  will  be  used 
for  preflight  data  pnf-ry  to  relieve  the  pilot 
from  a  lengthy  initialization  process. 

Inflight  dati  entry  will  he  managed  through 
the  Up  Front  Control  (UFC;  which  is  located 
just  belov’  true  HUD  Weapon  selection,  target 
designation,  and  display  management  functions 
are  pre  'ide i  through  standard  hands-on 

throttle  and  stick  switches,  or  by  switches 
located  around  the  perimeter  of  the  three 
multifunction  displays.  An  additional  method 
is  provided  by  a  touch  sensitive  overlay  for 
the  large  central  display  Pilot  working 
group  members  have  been  involved  in  a  series 
of  static  mockup,  rapid  prototyping,  and  part 
task  simulation  experiments  as  the  specific 
cockpit  layout  and  display  formats  have 
evolved  It  is  essential  to  achieve  a 
pilot- friendly  implementation  of  the  ICAAS 
technology 

TEST  AND..EVAUIAII91j 

Formal  high-fidelity  piloted  simulations  will  be  conducted  starting  in  late  1990  to  measure  the  combat 
performance  of  four  internetted  ICAAS  aircraft  engaging  up  to  sixteen  enemy  aircraft  An  orderly  buildup 
in  complexity  is  expected,  starting  with  two  blue  versus  two  red.  Blue  aircraft  use  cockpit  domes  which 
include  fully  configured  cockpits  with  ICAAS  controls,  displays,  information  processing,  and  system 
functions.  Red  forces  consist  of  up  to  four  manned  combat  stations  plus  digital  aircraft  models  Air  Force 
tactical  fighter  pilots  will  serve  as  test  subjects,  with  separate  blue  and  red  teams  A  mix  of  current 
and  advanced  threat  aircraft  are  used  to  represent  a  projected  1995  air  combat  environment 

Flight  testing  will  demonstrate  that  ICAAS  functions  can  be  Implemented  on  actual  fighter  aircraft 
using  real  sensors  in  real  flight  conditions.  An  F-15B  testbed  aircraft  is  being  modified  to  incorporate 
the  ICAAS  advanced  cockpit  configuration  and  system  functions,  including  IIC  and  ISC  computers  After 
initial  flight  testing  in  1991,  a  second  F-15  testbed  will  be  modified  in  1992.  An  intraflight  data  link 
will  be  installed  on  both  aircraft  to  enable  demonstration  of  the  cooperative  functions  of  ICAAS. 
Engagements  will  be  initiated  at  BVR  and  proceed  through  simulated  weapon  employments,  including  transition 
to  close-in  combat 


SUMMARY 

For  three  years,  the  ICAAS  program  has  been  developing  innovative  design  concepts  to  assist  tactical 
fighter  pilots  in  defeating  larger  forces  of  enemy  aircraft.  Avionics,  guidance,  and  control  functions  are 
being  improved  and  extensively  integrated  to  enhance  pilot  awareness  of  the  combat  situation,  provide 
decision  aiding  in  evaluating  offonsive/defensive  options,  support  the  pilot  with  precise  execution  of  his 
engagement  decisions,  and  coordinate  the  actions  of  individual  flight  members  into  an  effective  battle  team. 
Extensive  use  of  knowledge  based  systems  technology  and  emphasis  on  expert  systems  engineering  techniques 
provide  a  basis  for  significant  improvements  in  mission  effectiveness  compared  to  current  operational 
systems.  Efficient  information  management  combined  with  advanced  cockpit  display  concepts  allow  ICAAS  to 
effectively  Support  the  pilot  in  achieving  mission  objectives. 

A  specific  ICAAS  system  architecture  has  been  defined  to  achieve  real-time  performance  when  embedded 
in  a  fighter  aircraft.  New  flightworthy  computers  have  been  developed  based  on  advanced  32 -bit  RISC 
processors  The  DoD  high  order  Ada  software  language  is  being  used  to  implement  all  ICAAS  functions 
Integration  of  hardware  and  software  is  nearly  complete  in  preparation  for  formal  testing. 


Extensive  piloted  simulations  and  flight  test  experiments  will  be  performed  to  determine  the  mission 
effectiveness  of  ICAAS  technology  in  realistic  air  combat  engagement  conditions.  Test  and  operational 
fighter  pilots  will  be  used  as  test  subjects  to  confirm  that  the  system  provides  the  Intended  level  of 
support  and  meets  performance  goals.  Final  results  are  expected  to  be  available  in  late  1992.  Sufficient 
confidence  will  be  established  to  facilitate  technology  transition  to  future  fighters  or  upgrades  of  existing 
fighters 
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ABSTRACT 

Research  aircraft  have  become  increasingly  dependent  on  advanced  electronic  control  systems  to  accomplish 
program  goals.  These  aircraft  are  integrating  multiple  disciplines  to  improve  performance  and  satisfy  research 
objectives.  This  integration  is  being  accomplished  through  electronic  control  systems.  Because  of  the  number  of 
systems  involved  and  the  variety  of  engineering  disciplines,  systems  design  methods  and  information  management 
have  become  essential  to  program  success.  The  primary  objective  of  the  system  design/information  tool  for  air¬ 
craft  flight  control  system  is  to  help  transfer  flight  control  system  design  knowledge  to  the  flight  test  community. 
By  providing  all  of  the  design  information  and  covering  multiple  disciplines  in  a  structured,  graphical  manner, 
flight  control  systems  can  more  easily  be  understood  by  the  test  engineers.  This  will  provide  the  engineers  with 
the  information  needed  to  thoroughly  ground  test  the  system  and  thereby  reduce  the  likelihood  of  serious  design 
errors  surfacing  in  flight.  The  secondary  objective  is  to  apply  structured  design  techniques  to  all  of  the  design 
domains.  By  using  the  techniques  in  the  top  level  system  design  down  through  the  detailed  hardware  and  software 
designs,  it  is  hoped  that  fewer  design  anomalies  will  result  This  paper  will  first  review  the  flight  test  experiences 
of  three  highly  complex,  integrated  aircraft  programs:  the  X-29  forward-swept  wing,  the  advanced  fighter  tech¬ 
nology  integration  (AFTI)  F-16,  and  the  highly  maneuverable  aircraft  technology  (HiMAT)  program.  Significant 
operating  anomalies,  and  the  design  errors  which  cause  them,  will  be  examined  to  help  identify  what  funcuons  a 
system  design/information  tool  should  provide  to  assist  designers  in  avoiding  errors. 

1.  NOMENCLATURE 

AFTI  advanced  fighter  technology  integration 

AI  artificial  intelligence 

DFD  data  flow  diagram 

FCC  flight  control  computer 

FCS  flight  control  system 

FMEA  failure  modes  and  effects  analysis 

HARV  high-angle-of-attack  research  vehicle 

HiMAT  highly  maneuverable  aircraft  technology 

H/W  hardware 

ILS  instrument  landing  system 

KB  knowledge  base 

KBS  knowledge-based  system 

KCS  knowledge  capture  system 

KEE™  Knowlege  Engineering  Environment  (trade-mark  of  Intcllicorp,  Mountain  View,  CA) 

KR  knowledge  representation 

LE  leading  edge 

NWS  nosewheel  steering 

PLC  power  lever  control 

S/W  software 

TE  trailing  edge 

T/O  takeoff 

V  and  V  verification  and  validation 
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2.  BACKGROUND 

System  engineering  has  been  recognized  as  an  essential  clement  in  the  development  of  complex  systems.  The 
top-down  structured  approaches  to  system  engineering  have  been  slow  to  catch  on  because  of  a  lack  of  computer¬ 
ized  tools.  However,  recent  advances  in  personal  computers  and  new  software  (S/W)  tools  have  reestablished  the 
use  of  these  structured  system  engineering  methods  (ref.  1). 

The  system  design  aspects  of  the  system  design/  information  tool  expand  on  the  current  systems  engineering 
methods  by  1)  automatically  creating  a  knowledge  base  (KB)  of  the  processes,  data  flows,  and  externals;  and  2) 
including  functions  to  verify  consistency  in  design  requirements  unique  to  flight  crucial  control  systems. 

The  information  aspects  of  this  tool  address  the  need  to  provide  design  and  implcmentauon  informauon 
throughout  a  flight  control  system's  (FCS’s)  life  cycle,  and,  specifically,  to  the  test  engineers.  The  verification 
and  validation  (V  and  V)  effort  fi  r  the  digital  FCS  is  of  particular  concern  to  the  test  engineer.  Complete  V  and 

V  is  required  to  assure  flight  safety  and  requires  the  design  informauon  to  establish,  run,  and  analyze  the  V  and 

V  tests.  Problems  associated  with  V  and  V  have  caused  major  digital  FCS  developments  to  slip  by  as  much  as  18 
months  (ref.  2).  The  system  dcsign/informauon  tool  needs  to  include  the  flight  control  design  knowledge  and  its 
hardware  (H/W)  and  S/W  implementation. 

Figure  1  shows  a  typical  life  cycle  for  an  FCS  and  how  the  system  design/information  tool  would  support  all 
phases.  Shown  is  a  typical  life  cycle  for  an  FCS  and  how  the  life  cycle  phases  relate  to  the  system/information 
tool’s  capabilities.  Some  cf  the  current  tools  which  would  share  information  with  the  system  dcsign/information 
tool  are  shown  in  the  lower  half  of  the  figure. 

The  following  review  of  research  aircraft  and  the  unique  design  errors  that  were  found  shows  how  system 
complexity  can  hide  design  errors  from  even  the  most  experienced  engineers.  These  errors  reflect,  in  part,  the 
difficulty  of  adequately  communicating  the  system  design  details  to  the  test  engineers  in  the  muluple  disciplines. 
These  disciplines  include  flight  control  law  development,  H/W  design  and  test;  S/W  specification,  coding  and  test; 
system  integration  and  test;  and  flight  test  operations. 

2.1  X-29  Description  and  Airdata  Single-Point  Failure 

The  X-29A  technology  demonstrator  aircraft  is  an  experimental  vehicle  which  integrates  a  number  of  advanced 
technologies.  These  technologies  include  a  forward-swept  wing,  tailored  composite  wing  structure,  and  full  au¬ 
thority  digital  flight  control.  The  aircraft  is  also  highly  unstable  and  is  dependent  on  the  triplex  digital  FCS  for 
stability  and  handling  qualities  (ref.  3). 

The  FCS  feedback  gains  are  scheduled  using  airdata.  Airdata  errors  can  cause  incorrect  flight  control  gains  and 
loss  of  the  aircraft.  To  avoid  incorrect  gains,  the  X-29A  has  three  sources  of  airdata.  Redundancy  management 
S/W  takes  the  three  airdata  values,  detects  any  failures,  and  selects  a  value  to  be  used  in  the  control  law  calculations. 

After  flying  over  200  flights,  a  serious  design  error  in  the  redundancy  management  logic  was  found  during 
verification  of  a  new  release  of  flight  S/W  being  tested  in  ground-based  simulation.  The  error  was  attributed  to 
the  multidisciplinary  nature  of  the  system  and  had  been  in  the  flight  S/W  since  the  38th  flight.  A  lack  of  detailed 
understanding  about  the  interactions  between  the  airdata  system,  redundancy  management  S/W,  and  the  flight 
control  laws  allowed  for  the  design  error  to  occur  and  is  discussed  below. 

The  fault  detection  level  in  the  redundancy  management  S/W  was  set  at  a  large  value  because  of  errors,  such 
as  position  errors,  possible  between  the  probes  at  certain  flight  conditions  (fig.  2).  In  the  case  of  a  probe  failure, 
airdata  errors  as  large  as  the  fault  detection  level  were  allowed  to  pass  through  to  the  control  laws.  At  the  lower 
and  slower  end  of  the  flight  envelope,  a  fail-to-zcro  of  the  nose  probe  would  not  be  detected.  Simulations  have 
shown  that  this  single-point  failure  would  change  the  gains  to  the  point  that  the  aircraft  would  become  unstable 
and  depart.  For  over  162  flights  the  aircraft  was  at  risk  of  being  lost  because  of  a  single  airdata  failure.  The  system 
requirement  was  that  the  aircraft  be  operational  after  two  airdata  failures.  Until  a  subsequent  software  release 
corrected  the  problem,  the  aircraft  was  grounded. 

2.2  AFTI  F-16  Description  and  Flight  44  Anomaly 

The  advanced  fighter  technology  integration  F-16  program  investigated  the  integration  of  emerging  technolo¬ 
gies  into  an  advanced  fighter  aircraft.  The  AFTI's  three  major  technologies  investigated  were  (1)  flight-crucial 
digital  control,  (2)  decoupled  aircraft  flight  control,  and  (3)  integration  of  avionics  flight  control  and  pilots  display 
(ret.  2). 

The  AFTI  F- 16  flight  control  system  was  a  triplex,  asynchronous  digital  system.  The  asynchronous  architecture 
meant  that  input  signals  from  sensors  and  controllers  were  read  at  different  umes  into  the  three  computers  using 
a  high-speed  serial,  digital  data  link  (fig.  3).  Concerns  for  S/W  reliability  were  addressed  with  the  inclusion  of  a 
triplex,  analog-independent  backup  unit. 
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The  following  summarizes  an  in-flight  anomaly  which  occurred  on  flight  44  (ref.  2).  This  anomaly  was  the 
result  of  the  interaction  of  many  design  characteristics  and  a  unique  flight  condition.  The  characteristics  included 
asynchronous  computer  operation,  forward  integrators  in  the  control  laws,  and  output  redundancy  management 
S/W.  These  characteristics  coupled  with  a  unique  flight  condition  and  resulted  in  the  divergence  of  the  three  com¬ 
puters’  output  commands  to  the  control  surfaces.  The  redundancy  management  S/W  in  each  of  the  channels 
declared  the  other  two  channels  as  failed.  The  pilots  indication  of  this  apparent  simultaneous  failure  of  all  three 
computers  was  a  dual  fail  flight  control  light  in  the  cockpit.  The  end  result  of  this  in-flight  anomaly  was  that  the 
aircraft  safely  landed  on  what  was  effectively  a  single-string  flight  control  system,  even  though  no  actual  H/W 
failure  had  occurred. 

Like  the  X-29  example,  the  AFTI F-16  had  a  serious  design  error  resulting  from  the  lack  of  a  detailed  under¬ 
standing  of  the  interactions  between  the  many  different  disciplinary  areas.  In  this  case  the  design  error  was  not 
recognized  until  after  an  in-flight  anomaly  was  experienced. 

2 3  HiMAT  Design  and  Gear  Deployment  Anomaly 

The  HiMAT  demonstrator  was  a  remotely  piloted  research  vehicle  which  incorporated  such  advances  as  com¬ 
posite  structures,  aerr  elastic  tailoring,  reduced  static  stability,  and  digital  flight  control  (ref.  4).  The  aircraft  was 
remotely  piloted  because  the  technologies  represented  too  high  a  risk  for  a  manned  vehicle. 

The  HiMAT  was  flown  remotely  with  the  pilot  in  a  ground-based  cockpit  and  the  control  laws  calculated  in 
ground-based  computers.  Surface  commands  were  telemetered  to  the  aircraft  as  were  aircraft  sensor  data  which 
were  telemetered  to  the  ground  (fig.  4).  The  onboard  digital  flight  control  computers  were  dual  redundant  and 
processed  uplink  and  downlink  data.  In  the  case  of  a  complete  loss  of  the  dual  uplink  commands,  the  onboard 
system  acted  as  a  backup  FCS  capable  of  orbiting  the  aircraft  until  control  was  reestablished. 

An  anomaly  occurred  during  the  flight  test  program  which  resulted  in  the  aircraft  landing  with  its  landing 
skids  retracted.  However,  the  pilot  performed  an  excellent  landing  and  the  aircraft  was  not  seriously  damaged. 
The  anomaly  was  induced  by  a  single  failure  in  the  redundant  uplink  H/W.  The  onboard  redundancy  management 
S/W  identified  the  failure  and  allowed  for  continued  control  of  the  aircraft,  except  for  the  deployment  of  the 
landing  skids. 

The  anomaly  was  caused  by  a  timing  change  made  in  the  ground-based  system  and  the  onboard  S/W  for 
uplinking  the  gear  deployment  command.  This  change  coupled  with  the  onboard  failure  of  one  uplink  receiver  to 
cause  the  anomaly.  The  timing  change  was  thoroughly  tested  with  the  onboard  flight  S/W  for  unfailed  condiuons. 
However,  the  flight  S/W  operated  differently  when  an  uplink  failure  was  present.  This  critical  inhumation  about 
the  S/W  was  not  readily  available  to  the  flight  test  team. 

3.  REQUIREMENTS  FOR  A  SYSTEM  DESIGN/INFORMATION  TOOL 

These  brief  examples  demonstrate  that  system  complexity  is  overwhelming  the  individual’s  ability  to  under¬ 
stand  the  entire  system  and  the  interactions  that  can  take  place  between  the  different  functional  areas.  The  X-" 
example  illustrates  how  important  the  airdata  system  and  the  redundancy  management  S/W  design  information 
to  the  flight  control  designers  and  test  engineers.  Using  the  experience  gained  from  the  above  aeronautics  project, 
we  have  formulated  the  requirements  for  a  system  design/information  tool.  The  requirements  include: 

1.  A  system  design  capability  to  case  the  capture  of  design  information.  The  system  design  capability  will  pro¬ 
vide  a  graphical,  structured  method  for  designing  complex  systems.  It  wiil  help  the  designer  avoid  errors 
and  allow  the  capture  of  the  design  information  as  it  is  created.  Later  in  the  aircraft  development  this  infor¬ 
mation,  in  the  form  of  an  intelligent  documentation  system,  will  provide  information  to  the  test  engineers. 

2.  Online  documentadon  of  all  the  informadon  describing  an  FCS  and  the  relationships  betwee.'  Lt/erent  dis¬ 
ciplinary  informadon.  This  includes  H/W,  S/W,  redundancy  management,  and  flight  control  law  disciplines. 
The  test  engineer  can  then  easily  and  graphically  see  the  design  informadon  needed  to  qualify  the  system, 
thus  avoiding  the  in-flight  consequences  of  design  errors. 

3.  Expert  system  functions  to  help  analyze  the  reladonships  between  the  disciplines  and  uncover  where  un¬ 
wanted  interacdons  can  occur.  These  furtedons  cm  be  used  by  designers,  as  well  as  test  engineers,  to  assess 
the  system’s  operations  and  avoid  serious  design  errors. 

4.  Ability  to  perform  failure  modes  and  effects  analysis  (FMEA)  on  the  many  design  iteradons.  Cunendy, 
FMEA  is  only  performed  on  the  H/W,  not  on  the  system  as  a  whole.  Because  of  the  rime  required  to  perform 
an  FMEA,  the  FMEA  is  usually  performed  once  and  is  done  with  an  early  design  iteration.  The  inability  to 
analyze  the  final  design  raises  quesdons  of  the  FMEA’s  value.  Automated  FMEA  using  the  current  online 
design  is  one  example  of  a  capability  that  would  assist  designers  and  test  engineers  in  finding  serious  design 
errors  in  a  timely  manner. 


5.  Links  from  the  system  requirements  to  the  S/W  and  H/W  designs.  The  links  will  allow  the  system  require¬ 
ments  to  be  verified  against  the  proposed  implementation.  Verification  could  then  be  done  in  an  automated 
fashion,  prior  to  commuting  to  the  build  phase.  This  rapid  prototyping  concept  would  increase  the  chance 
of  finding  serious  design  errors  prior  to  flight  test. 

Currently,  some  system  design  tools  have  become  commercially  available,  but  they  do  not  address  the  needs 
of  flight-crucial  systems  and  only  create  conventional  databases  called  data  dicuonaries.  The  actual  H/W  and  S/W 
implementation  information  is  not  an  integral  part  of  these  tools. 

4.  DESCRIPTION  OF  THE  KNOWLEDGE-BASED  SYSTEM  DESIGN/INFORMATION 
TOOL 

The  following  section  will  review  the  work  accomplished  to  date  and  show  how  it  applies  to  the  larger  problem 
outlined  above.  The  methods  for  capturing  system  design  knowledge,  examples  of  what  can  be  done  using  this 
knowledge,  and  an  overview  of  the  structure  of  the  knowledge-based  system  (KBS)  will  be  discussed.  In  related 
work,  a  good  approach  to  design  knowledge  capture  for  the  space  station  can  be  found  in  Wechsler  (ref.  5). 

4.1  Focus 

The  effort  is  focused  on  the  development  of  a  generic  knowledge  capture  system  (KCS)  for  digital  FCSs,  which 
utilizes  mature  AI  technology.  The  KCS  is  being  used  to  capture  design  knowledge  for  the  NASA  high-angle- 
of-attack  research  vehicle  (HARV),  a  modified  F/A-18A  with  a  thrust-vectoring  capability.  The  primary  efforts 
to  date  have  been  focused  on  the  development  of  an  intelligent  documentation  system  for  the  system  and  H/W 
design  realms.  Examples  of  expert  analyses  that  can  be  accomplished  once  the  design  knowledge  is  captured  are 
described  in  the  sections  on  the  spin  recovery  system  and  nosewhecl  steering  behavioral  model. 

4.2  The  Knowledge  Capture  System 

A  major  portion  of  the  effort  for  the  KCS  has  been  devoted  to  the  development  of  a  knowledge  representation 
(KR)  which  is  tadored  to  the  specific  needs  of  the  FCS  problem  domain.  Four  domains  of  knowledge  have  been 
identified  within  the  FCS  problem  domain:  system  design,  H/W  design,  S/W  design,  and  utilities.  These  four 
domains  provide  the  flight  test  engineer  with  the  diverse  kinds  of  information  needed  and  the  relationships  which 
exist  between  them.  Each  domain  possesses  its  own  unique  KR. 

4 3  System  Design  Realm 

The  structured  analysts  methodology  is  used  to  describe  the  system  design  (ref.  6).  This  methodology  is  based 
on  a  top-down  hierarchical  decomposiuon  of  system  requirements  using  an  extremely  graphical  user  interface. 
The  decomposiuon  conUnues  until  the  requirements  are  given  with  an  adequate  degree  of  detail.  This  design 
methodology  creates  a  cleaner,  more  understandable  design. 

The  tool  creates  linked  hierarchical  trees  of  data  flows,  processes,  and  externals.  Each  node  in  the  process 
tree  represents  a  process  and  is  provided  with  a  process  description  and  other  unique  attributes  which  are  stored 
in  slots.  To  support  flight  control  system  design,  the  tool  stores  and  tracks  requirements  for  failure  probability 
and  mission  criticality.  In  addition,  external  agencies  and  data  flow  objects  are  identified  in  the  KB.  All  of  this 
information  is  depicted  graphically  in  data  flow  diagrams  (DFDs).  Figure  5  depicts  a  level  0  DFD. 

The  concepts  of  a  process,  an  external,  and  a  data  flow,  as  defined  by  structured  analysis,  are  identified  here  as 
graphical  objects  and  individually  represented  as  frames.  The  properties  of  the  process,  external,  data  store,  and 
data  flow  objects  are  stored  in  the  slots  of  the  individual  frames  associated  with  each  of  these  objects.  The  name 
of  the  process,  failure  probability,  and  data  flow  inputs  are  all  examples  of  slots.  The  nature  of  the  slot  values  can 
draw  from  the  full  spectrum  of  the  paradigms  supported  by  the  Knowledge  Engineering  Environment  (KEE™). 
Namely,  they  may  be  simple  values,  pointers  to  other  frames,  inherited  values,  active  values,  rules,  and  so  forth. 
This  KR  will  allow  the  users  to  perform  various  expert  analyses  of  the  system  design.  It  is  intended  that  the 
pointers,  which  are  stored  as  slot  values,  will  provide  access  to  the  related  H/W,  S/W,  and  utilities  implementation 
knowledge  stored  elsewhere. 

A  hierarchical  representation  scheme  is  used  for  each  of  the  three  types  of  objects  (processes,  externals,  and 
data  flows).  Each  of  these  three  hierarchies  forms  an  individual,  linked  KB.  In  each  case,  the  hierarchy  is  used  to 
allow  properties  to  be  inherited  and  to  identify  the  natural  linkage  between  individual  objects.  These  individual 
KBs  are  linked  with  pointers. 

In  a  typical  application  of  the  structured  analysis  methodology,  the  data  flow  diagrams  are  viewed  as  an  end 
object  In  this  KBS,  the  data  flow  diagrams  are  primarily  viewed  as  a  graphical  front  end  for  the  KCS.  Every  data 
flow  diagram  image  is  mouse  sensitive  and  possesses  its  own  menu  for  entering  and  accessing  knowledge.  Now 
the  test  engineer  can  graphically  see  the  relationships  between  systems,  rather  than  trying  to  infer  them  from  stacks 
of  written  text 
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4.4  Hardware  Design  Realm 

The  knowledge  representation  for  the  H/W  design  is  based  upon  the  hierarchical  block  diagrams  typically 
used  in  this  problem  domain.  The  nature  of  the  representation  is  similar,  although  not  identical,  to  the  structured 
analysis  methodology.  The  H/W  objects  are  represented  graphically  as  blocks,  and  these  objects  are  decomposed 
in  a  hierarchical  fashion  until  they  have  been  described  to  an  adequate  degree  of  detail. 

The  connectivity  between  these  H/W  objects  is  indicated  graphically  with  lines.  These  lines  may  represent 
various  H/W  connection  abstractions  such  as  data  buses,  a  flow  of  information,  or  a  form  of  control.  In  any 
case,  this  connectivity  can  be  represented  in  the  form  of  objects  of  a  specific  type  and  may  possess  a  hierarchi¬ 
cal  characteristic. 

The  concepts  of  H/W  and  their  connectivity  are  identified  in  the  KBS  as  being  objects  and  are  individually 
represented  as  frames.  The  properties  of  these  objects  are  stored  in  the  slots  of  the  individual  frames.  The  nature 
of  the  slot  values  and  the  hierarchical  relationship  of  tft$  frame  representation  is  the  same  as  that  provided  for  the 
system  design.  The  H/W  block  diagrams  serve  as  a  graphical  front  end  for  the  H/W  design  KB.  Figure  6  depicts 
a  top  level  H/W  block  diagram. 

4.5  Software  Design  Realm 

The  structured  analysis  methodology  is  also  used  to  describe  the  S/W  design.  The  long-range  plans  include 
placing  the  KCS  on  a  network  with  a  workstation  that  has  the  flight  code.  This  would  give  the  KCS  access  to  the 
flight  code  so  that  it  would  be  possible  to  pull  up  listings  of  the  flight  code  relevant  to  the  objects  defined  in  the 
S/W  design  data  flow  diagrams. 

4.6  Utilities  Design  Realm 

The  utilities  consist  of  the  electrical  power,  hydraulic  power,  and  environmental  control  systems  which  form 
an  infrastructure  for  the  FCS,  and  any  embedded  avionics  system  for  that  matter.  The  KR  for  this  realm  will 
utilize  the  structured  analysis  methodology  to  encode  the  design  knowledge.  It  will  also  utilize  the  block  diagram 
representation  described  previously  for  the  H/W  design  realm.  This  representation  will  be  used  to  encode  the 
implementation  knowledge. 

4.7  Authoring  and  Browsing  Mechanisms 

The  authoring  mechanisms  allow  the  user  to  create,  delete,  connect,  and  locate  the  user  interface  graphical 
images  with  mouse  and  menu  commands.  These  ordinary  capabilities  provide  typical  graphical  interface  features. 
More  importantly,  the  authoring  mechanisms  allow  the  user  to  properly  create,  delete,  rename,  and  connect  ob¬ 
jects  in  the  semantic  network.  The  objects  and  their  linkage  within  the  semantic  network  are  highly  structured 
and  canonical.  A  failure  to  conform  to  this  formal  structure  would  destroy  the  inheritance,  browsing,  and  reason¬ 
ing  functionality.  The  authoring  mechanisms  Jso  serve  to  enforce  the.  methodologies  which  have  been  deemed 
appropriate  for  the  individual  realms  within  the  KB. 

The  authoring  mechanisms  provide  mouse  and  menu  commands  for  inserting  slot  values  for  the  properties 
of  the  objects  in  the  KB.  These  mechanisms  include  an  editor  window  for  entenng  text  descriptions  and  popup 
selection  menus  for  properties  whose  slot  values  are  restricted  to  a  specific  set  of  legal  values.  In  some  cases,  the 
entry  of  slot  values  is  monitored  by  demons.  These  demons  actively  monitor  the  knowledge  entry  and  warn  the 
user  if  the  new  value  lies  outside  an  envelope  which  is  known  to  be  valid.  A  demon  is  used  to  verify  that  failure 
probability  requirements  are  kept  as  the  design  is  decomposed. 

The  browsing  mechanisms  allow  the  user  to  display  the  text  description,  hierarchical  relationships,  and  prop¬ 
erties  for  the  objects  which  have  been  entered  into  the  semantic  network.  This  information  is  accessed  through  the 
graphical,  menu-driven,  and  mouse-sensitive  user  interface.  This  interface  supports  a  random  access  to  the  KB. 
The  knowledge  representation  supports  a  browsing  strategy  similar  to  the  way  we,  as  humans,  pursue  problem 
solutions.  In  this  case,  the  network  structure  tends  to  guide  the  user  in  exploring  the  KB. 

4.8  A  Decomposition  of  the  Spin  Recovery  System 

The  HARV  flight  tests  will  include  research  flight  work  with  an  angle  of  attack  grea^r  than  55°.  For  safety  of 
flight  in  this  regime,  aspin  chute  recovery  system  has  been  added  to  theF/A-18A  research  aircraft.  The  following 
describes  a  decomposition  which  has  been  performed  on  the  spin  chute  recovery  system  using  the  KCS. 

Figures  7  and  8  show  the  hierarchical  decomposition  of  the  primary  system  that  will  deploy  a  spin  chute  for  a 
spin  recovery.  These  figures  depict  two  of  the  many  H/W  diagrams  involved  in  the  decomposition  of  the  spin  chute 
recovery  system.  Figure  7  includes  a  partial  display  of  the  hierarchy.  Figure  8  indicates  the  use  of  dual  abstractions. 

The  box/line  graphics  provide  an  abstraction  which  follows  directly  from  the  top  level  H/W  diagram  graph¬ 
ical  user  interface.  These  box/line  abstractions  are  linked  directly  to  the  objects  in  the  hierarchical  H/W  design 
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KB.  The  bit  map  graphical  depiction  of  the  circuit  diagram  has  been  added  to  clarify  the  user  interface  at  the 
component  level. 

The  two  abstractions  are  tied  together  with  bit-mapped  hot  spots.  Authoring  mechanisms  allow  the  user  to 
link  the  box/line  abstractions  (and  correspondingly  their  H/W  and  signal  flow  objects)  to  as  many  hot  spots  on  the 
bit  map  abstracuon  as  may  be  desired.  Browsing  mechanisms  allow  the  user  to  mouse  on  a  hot  spot  or  a  box/line 
abstraction.  Highlighting  indicates  the  correspondence  between  a  selected  hot  spot  and  its  box/linc  abstracuons. 
Similarly,  highlighting  indicates  the  correspondence  between  a  selected  H/W  object  (or  signal  flow  object)  and  the 
relevant  hot  spots. 

Further  work  in  this  area  will  be  concentrating  on  the  ability  to  take  detailed  H/W  diagrams  and  perform  failure 
modes  and  effects  analysis  of  the  systems  they  represent 

4.9  A  Behavioral  Model  for  the  Nosewhecl  Steering  System 

The  KBS  includes  dynamic  behavioral  models  to  aid  the  test  engineer  and  also  provides  for  rapid  prototyping 
of  the  system.  These  models  will  be  used  to  describe  the  system  operation  as  a  function  of  its  operating  modes. 
The  models  will  permit  the  user  to  interactively  enter  mode  commands  and  explore  their  impact  on  FCS  operation. 

The  behavioral  model  described  here  permits  the  user  to  issue  cockpit  mode  commands  to  a  nosewhee!  steering 
(NWS)  model  which  indicates  the  response  and  changes  in  the  aircraft  state.  The  model  is  based  upon  a  rule  set 
and  a  forward  chaining  paradigm.  A  dynamic  display  of  the  rules  and  their  execution  is  available  to  dynamically 
document  the  system  operation. 

The  NWS  is  a  secondary  control  system  within  the  FCS  which  is  only  operable  on  the  ground.  It  provides 
nosewhecl  angular  deflection  proportional  to  pedal  force  when  engaged.  There  arc  three  modes  of  operation:  off, 
low  gain,  and  high  gain.  The  desired  mode  is  selected  by  the  pilot,  with  switches  located  on  the  control  stick  grip, 
and  is  a  function  of  several  inputs,  such  as  wing  fold  and  weight  on  wheels.  The  NWS  switch  is  used  tor  NWS 
engagement  and  mode  control,  while  the  autopilot  switch  is  used  for  NWS  disengagement  on  the  ground. 

A  dynamic  interactive  display  (fig.  9)  is  provided  to  control  and  display  the  control  stick  s  witch  commands,  the 
NWS  system  status,  the  NWS  system  block  diagram,  and  the  relevant  F/A-  18A  aircraft  status.  The  display  window 
of  the  control  stick  switches  includes  a  control  stick  and  KEE™  acuve  images  for  the  NWS  and  autopilot  switches. 
This  window,  which  is  mouse  sensitive,  will  accept  switch  commands  in  an  identical  fashion  to  those  issued  by  the 
pilot  by  way  of  the  actual  aircraft  control  stick.  The  KEE™  active  images,  which  depict  the  NWS  switch  and  the 
autopilot  switch,  are  mouse  sensitive.  It  is  possible  to  issue  a  momentanly  depressed,  held  depressed,  or  released 
command  with  these  images. 

The  display  of  the  aircraft  status  is  also  mouse  sensitive.  It  is  possible  to  explore  the  NWS  logical  operation  as 
a  function  of  aircraft  power,  touchdown  status,  wing  fold  status,  and  launch  bar  status  by  mousing  the  appropriate 
active  image.  As  these  parameters  are  changed,  the  appropriate  operational  mode  is  dynamically  updated  and 
displayed  in  the  F/A-18A  operation  mode  window.  The  NWS-related  aircraft  operational  modes  are:  power  off,, 
wings  folded,  taxi,  takeoff  (T/O),  launch,  in  flight,  and  landing. 

A  rule-based  system  implements  the  NWS  mode  logic.  These  rules  are  activated  by  the  control  suck  switch 
commands  and  by  changes  in  the  aircraft  status  variables.  The  NWS  system  response  is  displayed  by  highlighting 
the  appropriate  mode  in  the  NWS  control  mode  status  window.  The  NWS  svstem  response  is  also  displayed  by 
highlighting  the  control  path  in  the  NWS  system  block  diagram  window.  It  is  possible  to  trace  the  rule  execution 
in  a  KEE™  dynamic  forward  chaining  execution  window.  The  rule  displays  arc  mouse  sensitive  and  permit  the 
user  to  display  a  selected  rule. 

4.10  Knowledge-Based  System  Structure 

The  KBS  is  coded  in  Common  Lisp,  utilizes  an  expert  system  shell  called  the  knowledge  engineering  environ¬ 
ment,  and  is  currently  under  development  on  a  Symbolics  machine  (fig.  10).  This  rapid  prototyping  environment 
has  been  utilized  for  the  development  of  a  KCS,  whicti  is  tailored  to  the  needs  of  the  FCS  problem  domain.  The 
KCS  can  l>e  ported  to  any  platform  which  is  supported  by  KEE™.  These  platforms  currently  include  the  Symbolics 
and  other  major  computing  environments. 

The  KCS  includes  authoring  mechanisms  which  enable  the  user  to  build  a  semantic  network  uniquely  appro¬ 
priate  to  a  particular  FCS.  The  KCS  also  includes  browsing  mechanisms  which  provide  access  to  the  semantic 
network  knowledge.  Rule-based  models  peiform  their  reasoning  on  the  objects  defined  in  the  semantic  network. 

The  semantic  network  is  composed  of  four  realms  of  knowledge:  the  FCS  system  design  realm,  the  FCS  H/W 
design  realm,  the  FCS  S/W  design  realm,  and  the  FCS  utilities  design  realm  (fig.  11).  Each  of  these  realms  is 
implemented  with  linked  hierarchical  networks  of  objects.  The  KBS  semantic  network  is  formed  by  linking  the  hi- 
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crarchical  networks  of  the  four  realms.  The  objects  are  individually  represented  with  a  frame-based  representation. 
Authoring  mechanisms  enable  the  user  to  define  a  semantic  network  of  FCS  objects  and  their  properties. 

The  semantic  network  of  FCS  objects  are  defined  in  an  environment  which  includes  an  inference  engine. 
Reasoning  functions  are  under  development  which  will  enable  the  user  to  view  and  analyze  the  objects  defined. 
Three  kinds  of  models  are  to  be  developed: 

1.  behavioral  models 

2.  failure  mode  and  effects  models 

3.  fault  tree  analysis  models 

The  behavioral  models  are  to  provide  a  dynamic  presentation  of  how  designated  objects  behave  as  a  function 
of  user  commands,  FCS  state  variables,  and  FCS  modes.  The  failure  mode  models  are  to  indicate  'he  loss  of 
functionality  associated  with  component  failures.  The  fault  tree  models  are  to  provide  a  diagnostic  capability  for 
the  loss  of  FCS  functionality.  This  diagnostic  capability  will  allow  the  user  to  identify  the  possible  causes  and  to 
help  isolate  the  actual  cause  of  the  loss  of  FCS  functionality. 

5.  CONCLUDING  REMARKS 

This  project  has  proven  to  be  an  ambitious  one.  The  roughly  three  man-years  of  effort  have  yielded  a  prototype 
which  promises  to  fulfill  the  objectives,  stated  earlier,  for  a  useful  flight-crucial  segment  of  the  high-angle-of-attack 
research  vehicle  flight  control  systen . 

So  far,  the  promises  of  artificial  intelligence  have  been  fulfilled.  It  has  been  possible  to  develop  a  knowledge 
capture  system  that  captures  the  flight  contro*  system  knowledge  in  a  form  which  is  tailored  to  the  problem  domain 
and  is  accessible  to  the  user  in  a  friendly  fashion.  Furthermore,  the  modeling  capability  has  proven  the  value  of 
defining  the  flight  control  system  objects  in  an  environment  with  an  inference  engine. 

The  system  and  hardware  design  realms  now  nave  a  working  functionality.  In  the  remaining  one  man-year  of 
effort,  it  is  projected  that  a  working  prototype,  capturing  knowledge  in  all  four  of  the  realms,  will  be  implemented. 
In  terms  of  computer  resource  requirements,  the  response  time  is  generally  adequate,  and  less  than  10  megabytes 
on  the  hard  disk  have  been  required  to  date  for  the  design  knowledge. 

Looking  to  the  future,  it  is  projected  that  this  prototype  provides  an  infrastructure  upon  which  a  full-scale,  fully 
operational  knowledge  capture  system  will  be  built  that  includes  design  aid  capabilities.  Longer  range  visions 
include  such  growth  possibilities  as  postflight  fault  diagnosis,  real-time  ground  support  of  flight  tests,  and  real¬ 
time  monitoring  in  the  cockpit.  The  complexity  of  today's  advanced  aircraft  demand  that  tools  such  as  this  be 
developed  and  utilized  throughout  the  life  cycle  to  assure  safe  and  efficient  flight  operations. 

6,  REFERENCES 

1.  Neubauer,  Chris,  Radloff,  B.,  and  Rhodes,  D.,  Optimizing  Systems  Engineering  Teams,  Methods,  andTools, 
AIAA  Paper  88-3870,  October  1988. 

2.  Mackall,  Dale  A.,  Development  and  Flight  Test  Experiences  With  a  Flight-Crucial  Digital  Control  System, 
NASA  TP-2857, 1988. 

3.  Chin,  J.,  Chacon,  V.,  and  Gera,  J.,  X -29A  Flight  Control  System  Performance  During  Flight  Test,  AIAA 
Paper  87-2878,  September  1987. 

4.  Evans,  Martha  B.  and  Schilling,  LJ.,  The  Role  of  Simulation  in  the  Development  and  Flight  Test  of  the 
HiMAT  Vehicle,  NASA  TM-84912,  April  1984. 

5.  Wechsler,  D.B.  and  Crouse,  K.R.,  An  Approach  to  Design  Knowledge  Capture  for  the  Space  Station,  NASA 
TM-89272, 1987. 

6.  DeMarco,  T.,  Structured  Analysis  and  System  Specification,  Prentice-Hall,  1978. 


54-8 


Life  cycle  for  flight  control  system 


System  design 

Hardware  and 
software  design 


Hardware  and  software 
Implementation 

Verification  and 
validation,  and 

_ _  ground  test 


Flight  test  operations 

_ Li . . . t  t _ 

System  design/information  tool  capabilities 


•  Structured  system  •  Import  data  from  CAD  •  Provide  information  to 
design  and  case  tools  flight  test  engineers 


•  Knowledge  base  of 

design 

•  Provisions  for  expert 

analysis 


•  Relate  system  design 

to  implementation 

•  Provide  failure  modes 
and  effects  analysis 
and  fault  tree  analysis 


•  Provide  expert  functions 
that  help  understand 
system  operation 

•  Support  ground  test 
and  flight  operations 


Current  tools  that  support  specific  design  areas  and  life  cycle  phases 


•  Flight  control  design  •  Computer-aided  design,  •  Simulation,  flight  control 
and  analysis  computer-aided  software  analysis  programs 

engineering,  source  code 
debugging  tools 

9129 

Figure  1.  Life  cycle  applications  for  the  system  design/information  tool. 
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Figure  2.  Summary  of  X-29  failure  condition  and  fault  detection  logic. 
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Figure  4.  HiMAT  control  system. 
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Figure  5.  F/A-18A  flight  control  system — top  level  data  flow  diagram  (level  0  DFD). 
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Figure  6.  F/A-18A  FCS — lop  level  hardware  diagram. 
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Figure  7.  F/A-18A  flight  control  system— spin  chute  deployment  system  hardware  diagram  (level  1  HWD). 
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Figure  8.  F/A-18A  flight  control  system— primary  spin  chute  deployment  circuit  hardware  diagram  (level  2 
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SUMMARY 


The  target  approach  over  large  distances  of  a  cruise  missile  type  vehicle,  which  Is  equipped  with  an  imaging  infrared  sensor  aided 
Inertial  navigation  system,  is  examined  in  simulation.  The  study  is  based  on  the  idea  of  using  the  same  image  sensor  for  navigation  update 
and  target  recognition  with  subsequent  tracking.  Knowledge  based  methods  are  found  to  play  a  key  role  in  solving  the  difficult  image 
interpretation  task  for  real  world  scenery.  The  final  extraction  of  navigation  data  from  the  processed  and  interpreted  IR-lmage  information,  and 
their  combination  with  the  inertial  sensor  data  is  based  on  conventional  optimization  and  filtering  techniques.  The  study  shows  that  the 
combined  information  leads  to  an  improvement  of  navigation  data.  Ritering  techniques  are  found  to  be  capable  of  quantitatively  estimating 
major  error  sources  inherent  to  the  gyros  and  accelerometers 


1.  INTRODUCTION 

The  position  and  the  course  of  a  missile  could  in  principle  be  determined  by  using  only  one  navigation  system.  Examples  are 
inertial,  ground  based  or  satellite  based  navigation.  However  by  combining  the  data  of  several  navigation  systems,  the  overall  precision  can  be 
Improved  [1] 

Besides  precision,  other  qualities  like  passiveness  (no  emitting  sources)  and  independence  of  external  aids  are  desirable  too  Inertial 
navigation  systems  (INS)  are  autonomous  in  this  sense.  But  the  components  of  an  INS,  the  gyros  and  accelerometers,  are  subject  to  a  number 
of  error  sources  This  leads  to  a  time-dependent  decrease  of  accuracy  for  location  and  orientation  measurements  Therefore,  prior  to  the 
termination  of  the  mission  updates  based  on  different  sources  are  required  from  time  to  time  It  13  even  possible  to  calibrate  the  INS  during  the 
mission. 


An  aid  for  the  INS  which  preserves  the  autonomy  and  which  is  passive  and  thus  not  detectable.  Is  navigation  based  on  Infrared 
Images  of  landmarks.  Combining  infrared  image  based  navigation  and  inertial  navigation  is  interesting  because  it  will  also  represent  an 
economic  solution.  The  same  image  sensor  Is  used  both  for  navigation  update  in  a  midcourse  guidance  phase,  and  target  recognition  with 
subsequent  tracking  in  a  terminal  homing  phase,  where  it  Is  needed  anyway 

Technically,  Inertial  and  Image  based  navigations  are  complementary  in  their  error  behaviour  The  short  term  stability  of  inertial 
systems  and  the  long  term  stability  of  the  image  based  navigation  complement  each  other.  A  plot  of  the  typical  time-dependences  of  the 
position  and  velocity  errors  of  an  aided  INS  is  shown  in  fig  1.  In  this  figure  the  updates  lead  to  substantial  reductions  of  the  errors  The 
magnitude  of  the  effect  depends  of  course  on  the  precision  of  the  aiding  navigation  source 
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Rg.l'  Error  propagation  of  an  aldad  navigation  system 

The  aim  of  fits  present  study  was  to  Investigate  methods  for  extracting  navigation  relevant  Information  from  Infrared  Images  and  to 
determine  how  this  Information  can  be  used  to  produce  calibration  effects  comp-rable  to  those  shown  In  tig.  1 


2. 


TASK  DESCRIPTION 


A  prerequisite  for  using  images  as  navigation  aids  is  a  map  Based  on  the  map,  one  can  calculate  the  position  of  certain  structures  in 
the  image  plane.  This  calculation  is  done  by  using  an  estimate  for  the  location  and  orientation  of  the  image  sensor,  resp  of  the  missile  itself. 
From  the  difference  of  the  expected  and  real  positions  of  the  structure  In  the  image,  corrections  for  the  location  and  orientation  of  the  missile 
can  be  derived  This  comparison  becomes  possible  if  the  structure  has  been  separated  from  the  background.  In  fig  2  the  expected  and 
extracted  real  positions  of  a  road  structure  are  plotted  on  top  of  the  IR-image.  The  structure  is  symbolically  stored  in  terms  of  straight  lines  and 
their  Intersections 


Fig  2:  Extracted  (full  lines)  and  expected  (dashed  lines)  road  structure  The  difference  in  location  and  orientation  reflects  the  INS 
navigation  error 

The  extraction  of  navigation  relevant  objects  from  images  relies  on  todays  abilities  of  image  interpretation  Solutions  In  this  area  are 
only  applicable  in  very  restricted  domains  (2)  If  one  considers  a  cruise  missile  type  carrier  flying  at  low  altitude,  additional  difficulties,  as  the 
visibility  of  objects,  arise.  Objects  can  be  partly  or  fully  hidden  due  to  hills  and  vegetation  Besides  that,  perspective  distortions  and  the 
influences  of  the  daytime  and  the  weather  must  be  handled  Therefore  the  recognition  algorithms  need  a  high  interpretative  power.  In  order  to 
achieve  the  interpretation  task,  certain  concepts  being  presently  applied  In  artificial  intelligence  models  have  been  analyzed,  adapted  and 
implemented  In  particular,  knowledge  based  search  techniques  turned  out  to  be  most  helpful  tools. 

By  an  appropriate  use  of  such  methods  in  combination  with  adaptod  preprocessing  techniques,  the  difficult  image  interpretation 
task  can  be  solved 


Flg.3.  An  example  image  to  be  interpreted  in  terms  of  navigation  relevant  data 
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3.  THE  KNOWLEDGE  BASED  IMAGE  INTERPRETATION  CONCEPT 


Some  necessary  properties  of  the  Image  Interpretation  system  can  be  derived  from  the  application  requirements  The  system  should  be: 
adaptable  to  different  types  of  targets  and  objects, relevant  to  navigation 

robust  against  Interference  factors  as  well  as  daytime  influences  end  weather 
capable  of  real  time  execution  on  a  suitable  hardware. 

Our  analysis  shows  that  the  following  procedure  responds  to  the  demands 

3.1  Preprocessing 

Figure  3  shows  an  Infrared  scene  that  la  well  suited  tor  navigation  update.  It  has  a  digitized  size  of  512x512  pixels.  The  aim  of 
preprocessing,  as  we  use  It.  Is  the  extraction  and  recording  ol  object  edges  Edge  extraction  techniques  allow  the  processing  of  low  contrasts 
Furthermore  edge  extraction  Is  Independent  of  daytime,  during  which  the  contrast  can  switch.  As  far  as  noise  is  concerned,  edge  extracting 
operators  can  be  conceived  such  that  local  continuity  Is  used  as  a  hint  for  the  existence  of  an  edge  (3).  On  the  whole,  these  techniques  seem 
to  be  essential  to  preserve  robustness  at  the  early  processing  stages 

Edges  are  recorded  as  objects  with  as  many  attributes  as  possible.  We  think  that  It  Is  crucial  for  the  application  of  knowledge  based 
Interpretation  techniques  that  as  little  Information  as  possible  Is  lost  during  preprocessing.  On  the  other  hand,  the  results  mus!  be  represented 
in  terms  of  data  structures,  that  can  be  used  as  Input  to  the  Interpretation  programs.  Therefore  the  result  of  Image  preprocessing  Is  not  another 
filtered  Image  but  a  series  of  edge  elements  from  which  the  original  image  could  approximately  be  reproduced.  In  fig.  4  the  edge  elements  of 
fig.  3  are  shown.  They  are  represented  as  straight  line  segments.  In  fig.  5  edge  segments  from  the  boundaries  of  contrast  stripes  have  been 
combined  [4],  Thus  we  get  a  representation  of  Tines'  lhat  are  present  In  the  Image  In  fig  5  lines  that  are  brighter  than  the  surrounding  back¬ 
ground  are  shown  The  dark  lines  are  also  recorded.  The  road  that  runs  through  the  village  can  already  be  located 


Fig  41  Edge  elements  of  part  of  the  image  shown  in  fig  3  The  straight  line  segments  lie  on  the  boundary  of  positive  contrast  domains 
3.2  Interpretation 

The  method  for  Interpreting  the  preprocessed  Image  data  In  terms  of  meaningful  objects.  Is  based  on  abstract  models  of  these  objects  (5) 
The  shape  of  an  extended  object  is  our  primary  criterion  for  recognition  This  Is  however  not  the  only  possible  Interpretation  paradigm  Texture 
could  be  used  as  well.  'Shape  recognition"  turns  out  to  be  a  very  Important  method,  because  of  the  robustness  requirement  Objects  may  be 
partly  Invisible  and  still  show  some  critical  shape  features  Shape  models  are  implemented  with  the  help  of  rule  based  programming.  It  Is 
easily  possible  to  switch  between  different  rule  subsets  and  interpret  Images  In  terms  of  different  expected  structures. 
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Rg  5  Symbolic  representation  of  the  lines  of  part  of  the  image  of  fig  3 

Due  to  the  fact  that  objects  appear  differently  on  the  map  and  in  the  image,  a  modelling,  which  is  In  principle  Independent  of  the 
perspective  and  other  distortions  Is  required 

Therefore  the  Interpreting  rules  represent  Invariants  In  the  relations  between  object  parts  Such  Invariants  can  be  related  to  proximity, 
number,  symmetry  etc 

A  simple  example  should  illustrate  the  basic  method.  Figure  6  shows  an  abstract  model  of  the  prominent  road  structure  of  fig  3  The 
aim  is  to  identify  certain  straight  line  segments  of  fig  5  as  belonging  to  the  road 


Rg.6.  a)  Map  section;  b)  Object  model,  c)  Object  decomposition  and  definition  of  sesrch  levels 

In  this  example  we  shall  use  an  Important  characteristic  that  is  not  changed  by  the  transformations  frcm  the  map  to  the  Image  plane. 
It  is  the  convexity  of  the  example  structure  If  we  define  a  sense  of  rotation  as  Indicated  In  fig  6b,  distinctive  properties  for  the  three  straight 
line  segments  can  be  derived.  Taking  the  line  segments  two  by  two,  there  Is  always  an  Incoming  and  an  outgoing  line  (fig.  6c).  One  can  only 
differentiate  between  Incoming  and  outgoing  lines  in  the  light  of  the  predefined  rotation  sense  and  the  fact  that  two  lines  form  a  corner  that  Is 
part  of  the  structure.  This  ,ery  simple  example  ought  to  show  how  certain  shape  properties  can  be  described  In  terms  of  relations  between 
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parts  K  is  seen  from  figs.  6b  and  6c  that  several  Interpretation  levels  must  be  Introduced.  Beginning  at  the  line  segment  level  comers 
are  built  which  are  subsequently  combined  to  the  3-llne  convex  structure,  which  is  part  of  a  polygon 

The  stepwise  approach  to  Interpretation  is  similar  to  the  'blackboard*  procedure  (6)  and  to  dynamic  programming.  Tha  efficiency  of 
the  search  relies  on  the  introduction  of  tha  steps  and  tha  use  of  adequate  knowledge  which  acts  as  a  filter  for  the  creation  of  objects  on  the 
next  higher  level.  This  approach  Is  different  from  a  hypotheses-creatlng-and-testing  approach.  It  Is  more  efficient  and  uses  leas  memory. 

However  using  shape  Invariants  for  object  recognition  Is  not  sufficient.  On  the  highest  levels  of  Interpretation  there  may  still  exist 
several  similar  objects,  which  cannot  be  discerned  by  the  above  procedure.  H  this  Is  the  case,  knowledge  about  tha  expected  approximate  size 
can  be  used  to  discriminate  between  tha  candidates.  This  knowledge  Is  derived  from  the  navigation  parameters  given  by  tha  INS.  It  Is  vague  In 
the  sense  that  the  system  must  be  aware  of  the  Inaccuracy  of  the  INS  data. 

Fig.  7  shows  8  remaining  candidates  for  the  road  structure  discussed  above,  Some  vague  Information  on  the  expected  orientation  of 
tha  structure  has  bean  used.  Further  knowledge,  concerning  tha  approximate  size,  already  leads  to  a  unique  solution.  It  turns  out  that  tha 
candidate,  which  Is  largest  In  size  Is  most  compatible  with  tha  expectations.  This  last  step  Illustrates  an  Important  feature  of  our  method.  Tha 
INS  can  cooperate  with  the  knowledge  based  Interpretation  system. 


Flg.7  Remaining  candidates  after  applying  the  convexity  constraint  and  vague  orientation  knowledge 

4.  DERIVING  NAVIGATION  DATA  FROM  IMAGE  OBJECTS 

isolated  points,  Intersection  points  Bnd  straight  line  orientations  are  the  quantities  that  are  mainly  used  for  deriving  navigation  data. 
Surface  moment  parameters  could  also  be  considered  The  ubserved  difference  between  the  expected  and  measured  values  for  these 
quantities  can  he  expressed  In  terms  of  corrections  to  the  INS  navigation  parameters.  These  parameters  consist  of  the  location  and  orientation 
In  space  This  leads,  among  others,  to  six  parameters  to  be  determined.  In  the  example  case  of  chapter  3  it  Is  possible  to  determine  these  6 
parameters  from  the  Information  obtained  In  cne  image  frame.  In  general  filtering  techniques  have  to  be  applied,  because  the  relative  motion 
of  the  missile  between  frames  must  be  taken  Into  account.  It  is  obtained  from  the  INS,  as  the  error  that  accumulates  during  the  time  elapsed 
between  successive  frames,  can  be  neglected.  Non-linear  filtering  techniques  have  been  applied  In  order  to  derive  the  navigation  parameters. 

5.  UPDATING  THE  INS 

The  objective  of  an  Inertial  navigation  system  update  Is  to  improve  the  navigational  data  such  as  position  and  velocity.  Furthermore, 
the  updates  can  be  used  to  estimate  Inertial  sensor  errors  If  sensor  error  compensation  is  applied,  the  accuracy  of  the  system  can  be 
Improved  eve'  more.  The  navigation  uodates  are  supplied  by  the  Image  analysis  system  (IAS)  The  update  Information  consists  of  a  jrrent 
position  vector  of  the  missile,  whl'h  Is  made  available  to  the  INS 

The  update  procedure  Is  based  on  error  models  of  the  INS  and  IAS.  An  optimal  filter,  constructed  with  the  help  of  those  error  models, 
combines  the  continuous  navigation  data  from  the  INS  with  the  position  fixes  from  the  IAS  In  order  to  calculate  the  navigation  ar.d  sensor 
errors.  A  correction  algorithm  applies  sensor  calibration  and  resets  the  Alter  parameters  (see  figure  8). 

5.1  INS  arror  sources 

In  the  study  we  considered  a  strapdown  Inertial  system,  The  accuracy  oi  such  a  system  Is  Influenced  by 
the  Initialization  errors, 
the  computational  errors, 
the  direct  sensor  errors, 

sensor  errors  resulting  from  the  dynamic  environment 
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Th*  Initialization  errors  relate  to  the  uncertainties  of  the  location,  velocity  and  orientation  at  the  start.  These  uncertainties  are 
negligible  If  the  missile  Is  launched  from  the  ground.  Numeric  precision  is  particular  Important  with  strapdown  systems.  The  three  orientation 
angles  must  be  continuously  recalculated  from  the  signals  of  the  gyros.  The  accuracy  of  the  result  depends  on  the  numeric  procedure.  In 
general  one  tries  to  minimize  the  relative  Influence  of  this  error  source. 

By  direct  sensor  errors  one  understands  the  imperfections  of  the  inertial  Instruments  (gyros  and  accelerometers).  Their  Influence  can 
be  disastrous,  because  they  are  Integrated  over  time.  Position  errors  due  to  the  gyro  drifts  can  evolve  with  the  third  power  of  time.  It  Is  possible 
to  estimate  the  sensor  errors  using  Kalman-filtering  and  to  apply  compensations. 

In  strapdown  Inertial  systems  tome  errors  depend  on  the  dyn'  ■  environment.  For  example  the  errors  Induced  by  the  gyro  scale 
factors  Increase  with  increasing  angular  rates  of  the  missile  body  Depending  on  the  magnitude  of  the  angular  rates,  it  Is  also  possible  to 
estimate  scale  factor  errors 

The  Influence  of  the  different  kinds  of  error  sources  has  been  determined  In  simulation.  As  a  result  the  error  propagation  Is  based  on 
a  simplified  model.  It  turns  out  that  the  dominant  role  is  played  by  the  accelerometer  bias. 


( 
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Fig  8  Data  flow  diagram  for  the  INS  update 

S.2  Estimating  the  INS  errors 

In  the  block  diagram  of  fig  8  the  INS  updating  procedure  is  shown  The  image  analysis  along  the  flight  trajectory  provides  a  location 
vector  r(.  The  difference  between  the  INS-value  r„  and  r(  is  the  measurable  quantity  which  is  fed  into  the  Kalman-filter.  Here  a  new  state 
consisting  of  location  error,  velocity  error  and  accelerometer  bias  Is  estimated.  Via  a  correction  algorithm  the  filtering  results  are  adapted  for 
the  navigation  processor  and  the  filter  itself 

6.  RESULTS  AND  CONCLUSIONS 

Computer  flights  have  been  performed  using  real  world  scenery  For  this  purpose  an  object  oriented  database  for  the  storage  of 
generic  ob|ect  data,  as  shown  in  fig  6b,  has  been  designed.  In  this  database  the  navigation  relevant  features  of  objects  and  structures  are 
stored.  It  must  not  be  confounded  with  the  knowledge  base  used  for  recognition,  although  one  could  think  of  combining  the  two  kind  of  data 
in  a  later  stage. 

From  the  simulated  flights  the  following  conclusions  could  be  drawn-  Using  knowledge  based  techniques,  it  is  possible  to  implement 
an  Image  processing  concept,  which  allows  the  extraction  of  navigahon  relevant  data  The  accuracy  of  the  data  is  sufficient  for  supporting  the 
Inertial  navigation  system.  The  Integration  can  be  done  by  using  filtering  techniques  Orientation  data  can  also  be  derived  from  the  Image 
data,  but  It  has  not  yet  been  taken  into  account.  Real  time  execution  seems  possible  on  a  suitable  processor  hardware 
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